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Executive Summary 



This report presents the results from a seven-month study of successful schools in California 
performed by the American Institutes for Research (AIR). This study is part of a larger group of 
studies coordinated through Stanford University and funded by the Bill and Melinda Gates 
Foundation, the William and Flora Hewlett Foundation, the James Irvine Foundation, and the 
Stuart Foundation. 

The study explored some of the concepts underlying the “successful schools” approach to 
defining education adequacy and considered their implications for analyzing educational 
adequacy in California. The overall purpose of the paper is summarized in the following research 
questions: 

■ How has the successful schools approach been used to consider educational adequacy? 

■ What are the strengths and weaknesses of these alternative applications? 

■ How might successful schools be identified in the state? 

■ What resource differences are observed or reported by these schools? 

■ Can we predict academic performance by levels of resources and types of students 
enrolled? 

■ Is there any evidence that successful schools use their resources more efficiently? 

■ What other factors appear related to their success? 

■ What are the implications of these findings for defining education adequacy in 
California? 

The successful schools approach seeks to determine the cost of the education needed to reach a 
specified level of educational outcomes by identifying districts achieving these outcomes and 
determining how much they are spending. This study sought to improve on this basic approach 
by selecting schools that have been consistently performing at a higher level than the one 
predicted by their demographics, rather than selecting successful schools that are above an 
absolute level of performance in a given year or over a given period of time. We analyzed these 
schools that were “beating the odds” (BTO) with regard to student achievement and compared 
them to low-performing (LP) schools — schools that had been performing at a lower level than 
predicted by their demographics. We also conducted telephone interviews with a total of 23 
schools from both groups in an attempt to understand their resource allocation practices and to 
identify common themes in the factors principals deemed necessary for success. 

A major premise underlying all approaches to education adequacy is that success is directly 
linked to the resources available. This study explored this assumption by looking for evidence to 
confirm or disprove the idea that resource quantities constitute the primary distinguishing factor 
between successful schools and all others. Thus, the first question was whether BTO schools 
have more resources than LP or other schools. The second question was whether BTO schools 
are using their resources in more efficient ways than other schools in the state. We did not intend 
to estimate an “adequate” level of spending for all schools in California based on these BTO 
schools, but instead shed some light on whether successful schools have more resources and/or 
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use their available resources in more efficient ways than other schools in California, and the 
extent to which the total amount or mix of resources or their unique use appears to explain their 
exceptional academic achievement levels. 

One striking finding of our study is that only a small number of schools (103 schools) met all the 
requirements to be considered a beating-the-odds school. Beating-the-odds schools were those 
that were outperforming similar schools during a four- year period (three years in the case of high 
schools). The same was true for low-performing schools. We found considerable instability in 
their test score results: schools could outperform one year, and the next perform as expected or 
underperform. For example, 365 elementary schools were beating the odds during 2002 and 307 
schools were beating the odds during 2003; however, only 61 elementary schools were beating 
the odds every year over the 2002-05 time period. 

We focused our resource analyses predominantly on the allocation of personnel resources 
because these are so fundamental to the educational process and are such a large component of 
educational spending. We found some differences in staffing between BTO, LP, and other 
California schools, but these did not appear to be substantial. The most important difference 
appears to be the level of experience that administrators hold: elementary BTO administrators 
have more experience than their colleagues in other public schools. Fewer LP school 
administrators have advanced degrees, and they have less experience in education-related work. 
In addition, it is worthy of note that LP schools have less-qualified teachers (measured by 
education and experience level), and more staff focused on providing pupil support (e.g., 
counselors), which may be a reflection of the higher needs at these schools 

Do BTO Schools Have More Resources than Other Schools in the State? 

The results of a multivariate OLS regression model designed to explore how personnel resources 
in BTO and LP schools differed in comparison to other public schools provided no statistical 
evidence that BTO and LP schools have different quantities of certified and classified personnel 
when compared to other public schools. Also, BTO schools do not have smaller classes in grades 
1 to 5 compared to other public schools (BTO schools do have two fewer students in their 
kindergarten classrooms). The education and experience levels of teachers do not differ in 
comparison to other public schools, but elementary BTO administrators are more experienced 
than their counterparts in other schools. Both the level of experience of teachers and the 
administrators’ educational attainment in elementary LP schools are lower than in BTO schools 
and lower than the state average. Further, teacher education and experience is lower in LP 
middle and high schools than in other public schools. 

BTO elementary schools have a higher proportion of staff in administrative positions and have a 
significantly lower percentage of teachers with tenure when compared with other public schools. 
LP schools have significantly lower percentages of teachers holding full credentials, but they 
have a higher proportion of staff in pupil support assignments when compared to other public 
schools. 

Costing Out 

We also costed out BTO and LP schools in order to compare their levels of spending to one 
another. We found that, on average, beating-the-odds schools appear to spend slightly less than 
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low-performing schools per student ($7,799 vs. $8,021; the statewide average is $7,523). For 
both BTO and LP schools, total per pupil spending is positively correlated with poverty — high- 
poverty schools tend to show higher per pupil spending. In addition, high-poverty BTO schools 
show higher per pupil spending than high-poverty LP schools, with the reverse true for low- 
poverty schools. 

Can We Predict the Unusually High Performance of BTO Schools by their Level of Resources? 
We used an academic production model in an attempt to relate resource allocation practices and 
student demographic characteristics to differences in student academic attainment. We found that 
available measures of resources and student characteristics did not appear statistically related to 
the unusually high academic performance of BTO schools. It is possible that the factors that 
make schools successful are unobservable characteristics, or are not captured by current 
statewide databases. For example, indicators associated with school leadership, teachers’ 
planning time, and teacher and principal turnover are not uniformly available to conduct a 
comprehensive analysis for all schools in the state. 

Are BTO Schools More Efficient? 

We also addressed the issue of efficiency, asking whether there is room to reduce schools’ level 
of spending without altering their student achievement level. To do so, we concentrated on the 
optimal ratio between teacher experience and education. We found that the actual levels of 
teacher education and teacher experience in BTO schools are closer to the optimum predicted by 
the model, or in other words, BTO schools appear to be operating at a more efficient level than 
LP schools in regard to their teacher characteristics. Our findings suggest that if BTO and LP 
schools were to move to the optimum level predicted from our model, they would gain in 
efficiency by reducing spending on teaching staff. It is important to keep in mind that in the case 
of LP schools this would result in a gain in efficiency, but not in an improvement of their 
academic performance level. 

What Did We Find In Our Telephone Interviews? 

Knowing that BTO schools do not appear to have more resources than other schools seems an 
important finding. Still, it begs the question of what does make a difference in BTO schools. 
Statewide data do not provide sufficient depth to provide much insight into one of the primary 
research questions for this study: What resource allocation patterns, instructional practices, 
programmatic leadership, or professional development opportunities are observed or reported by 
schools as being related to their academic success? 

In an attempt to dig deeper into the factors that seem to have made a difference for BTO schools, 
we conducted telephone interviews with 18 BTO principals and five comparison schools’ 
principals. From our interviews, it seems clear that at least among these schools there is not a 
single key to academic success. It is not even clear that there is a single combination of factors, 
or a single recipe. In many ways, the combination of factors said to be major contributors to the 
success of the BTO schools seemed unique to each school. On the other hand, some factors were 
mentioned frequently enough to emerge as themes. The major factors identified by the 
respondents were: 1) the existence of high-quality teachers and staff, 2) implementation of a 
standards-based curriculum, and 3) coherent instruction. 
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There is a elose relationship between these three major faetors. High-quality teachers are 
responsible for delivering the content of the curriculum to their students. They are held 
accountable for ensuring that students understand the main content areas that are covered in the 
curriculum. In turn, the curriculum is used to inform the instruction. Finally, the instruction is the 
way in which the curriculum is delivered to the students. These three components all contribute 
toward having high-performing students. 

In addition, seven other factors were mentioned by the principals as contributing to school 
success by influencing the three major factors mentioned above. Teacher support and training, 
which refers to any professional development opportunities or support, increases the quality of 
teachers. Control over hiring ensures that new teachers have the training and skill set necessary 
to meet the expectations of the school. Effectiveness in removing teachers gives principals the 
ability to replace teachers that are consistently unable to meet expectations. Teacher 
collaboration time improves teacher quality by allowing them to discuss the curriculum, plan out 
their instructional strategies, and be aligned in their instructional practices and vision for the 
school. Assessment data informs instruction by allowing teachers to identify areas in which 
students are still struggling and areas which the students have mastered. 

The three other components that were also mentioned are more directly linked to the students: 
interventions and/or student services, which can either be additional help to students that are 
struggling or services that are intended to improve achievement; parental involvement, which 
provides extra encouragement from home and informs parents about the education of their 
students; and high expectations for students, which encourage them to challenge themselves 
academically and feel confident in their ability to perform at a high level. 

School and district leaders and external consultants, which have the potential to influence school 
performance, were also mentioned by our respondents. While only a few schools have hired an 
external consultant, every school has an identifiable leadership structure and some degree of 
collaboration with its district. The most common leadership structure is centered on a dominant 
principal, seen in 1 1 of the 18 BTO schools included in this study. 

Furthermore, of the BTO schools interviewed by phone whose API scores were below 800 (14 
schools), nine felt that an API of 800 was an obtainable goal for the year 2013. Of the 
administrators that saw 800 as obtainable, none saw more funding as necessary, but the majority 
stated that money would be helpful in achieving the goal. A few communicated satisfaction with 
their present level of funding. However, LP school administrators expressed frustration about 
lacking the funding to implement the programs that would allow their schools to reach the 2013 
API goal. 

What Can We Conclude? 

In short, what we have found is that the answer to success across the BTO schools in this study is 
complex. It is not simply more resources or the application of a certain recipe in regard to 
resource allocation practices. However, what we found in this analysis is not new. Our findings 
are similar in many ways to those of other research that have investigated the differences 
between relatively successful and unsuccessful schools. And although the linkage between the 
existence of high-quality teachers and school success seems somewhat obvious, the findings 
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from this report suggest that such staff can be attracted to schools with high concentrations of 
students with special needs (e.g., students in poverty and English learners). To attract them we 
need to create an environment in which they believe they have a chance to be successful. Some 
resource considerations in relation to this may be stable leadership, district support, and 
discretion at the local level in regard to being able to attract and retain other high-quality teachers 
and to remove those who prove to be ineffective. 

From an overall adequacy perspective, our findings seem to challenge the basic underlying 
premise that the primary element that is lacking in regard to realizing state outcome goals is 
directly related to the quantities (or even to the attributes) of educational resources. It may be 
that simply adding more resources will be unlikely to make a difference in regard to school 
performance. This would suggest a somewhat different conceptualization of the adequacy 
question than has been commonly employed. 

Undoubtedly, there are certain minimum levels of resources that are imperative for school 
success. Beyond this, however, we may need to broaden this perspective to begin specifying 
adequate conditions for schools’ success. To examine this further, we may need resource 
measures that, at least in California, we do not currently have. For example, we do not have 
measures of the stability of leadership and instructional staff at the school. We do not know the 
degree to which there is latitude for schools to select, retain, and remove teachers as needed to 
ensure a “quality” staff. We have insufficient measures to ensure district support for high needs 
schools — e.g., ensuring that they have at least equal resources in comparison to all schools in the 
district. 

In summary, identifying and analyzing BTO schools has provided insight into our overall 
conceptualization of educational adequacy. The basic underlying premise for adequacy as it has 
been largely defined and applied is that we simply need a better understanding of the levels of 
resources needed to reach a specified educational outcome standard. The analyses in this report 
suggest that at least for the pool of schools realizing this level of success at a much greater rate 
than their counterparts, traditional resource measures do not seem to be what are making the 
difference. 

This does not lead to the conclusion that resources do not matter. All of these schools do have 
resources at a certain specified level; none would likely say that they could continue to perform 
at this level with less, and most would probably argue for more. Perhaps existing adequacy 
frameworks would benefit from considering more broadly the mix of school-wide staff 
attributes, as well as counts of staff and non-personnel resources, needed in a school to be truly 
adequate for success. The state can further this agenda through more comprehensive data 
collection in regard to the broader sets of attributes and performance measures that are needed to 
better understand the full resource implications of schools’ success. 
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Chapter I. Introduction and Overview 



Introduction 

At the beginning of 2006, Stanford University eontraeted with the American Institutes for 
Research (AIR) for a seven-month study of successful schools in California. This study is part 
of “Getting Down to Facts,” a research project of more than 20 studies designed to provide 
California’s policy makers and other education stakeholders with information to assist in raising 
student achievement and repositioning California as an education leader. The group of studies is 
funded by the Bill and Melinda Gates Foundation, the William and Flora Hewlett Foundation, 
the James Irvine Foundation, and the Stuart Foundation. 

The “successful schools” approach is one of four dominant approaches that have been used to 
attempt to measure education adequacy. This approach has been used in a number of studies 
across the states. A listing and overview of these prior studies is contained in Chapter 2. 

This study explores some of the concepts underlying the successful schools approach to defining 
education adequacy and considers their implications for the consideration of adequacy in 
California. The overall purpose of the paper is summarized in the following research questions: 

■ How has the successful schools approach been used to consider educational adequacy? 

■ What are the strengths and weaknesses of these alternative applications? 

■ How might successful schools be identified in the state? 

■ What resource differences are observed or reported by these schools? 

■ Is there any evidence that successful schools use their resources more efficiently? 

■ Can we predict academic performance by levels of resources and types of students 
enrolled? 

■ What other factors appear related to their success? 

■ What are the implications of these findings for defining education adequacy in 
California? 

A simplified way to state the basic rationale underlying this approach is as follows: an effective 
way to determine the cost of the education needed to reach a specified level of educational 
outcomes is to identify schools (districts) achieving these desired outcomes and to determine 
how much they are spending. This approach is appealing in its apparent simplicity. However, a 
major concern in determining education adequacy in this manner is the criteria used to select 
these schools. For example, in California, a simple definition of the schools that might be 
considered successful under this approach is those scoring 800 on the state’s Academic 
Performance Index (API). This is the target set by the State Board of Education for all schools 
under the state accountability system. 

As shown in Exhibit 1.1, student characteristics in schools scoring at or above 800 API are very 
different from those scoring below this level. Given that their percentages of students in poverty 



American Institutes for Research 



Page 1 




Successful California Schools in the Context of Educational Adequacy 



and English learners are less than one-third of those found in the average state school, the level 
of resources producing “success” in these schools will not necessarily be sufficient for ah of the 
schools in the state. 



Exhibit 1.1. Ali California Schools as Compared to Those at or Above 800 API 





School 

Level 


N 


% Free and 
Reduced 
Price Lunch 


% English 
Learners 


Enrollment 




Elementary 


1,401 


17.0 


8.7 


541 


Schools at or 


Middle 


210 


11.4 


4.9 


885 


above 800 API 


High 


71 


10.3 


3.6 


1,627 




Average 


1,682 


16.0 


8.0 


630 




Elementary 


5,582 


53.6 


27.5 


562 


All schools 


Middle 


1,274 


47.3 


20.3 


927 


High 


1,159 


35.6 


14.9 


1,540 




Average 


8,015 


50.1 


24.7 


761 



For this reason, an important dimension that distinguishes how this methodology has been 
applied pertains to whether an attempt has been made to control for the characteristics of the 
students enrolled in these “successful” schools and/or districts. When controls are not used, 
“successful” schools could simply be those with the lowest percentages of students with special 
needs (e.g., those in poverty or who are English learners). 

When this approach is used to produce an estimate of education adequacy to be applied to 
schools and districts with varying student populations, it is important to carefully consider the 
impact of student characteristics, as well as the assumption that it is possible to define adequate 
levels of spending simply from what is found in “successful schools.” Accordingly, this report 
presents a more complex methodology for identifying successful schools, explores measures of 
their resources, and attempts to identify a broader mix of factors that may be leading to success. 

The analysis presented in this report factors in student characteristics in the selection of 
successful schools. Rather than defining successful schools in an absolute sense as in the 
example above, this alternative approach focuses on “beating the odds” or BTO schools. This 
approach identifies schools that are producing substantially higher educational outcomes than 
expected given the demographic composition of their students. It arguably provides a better 
indication of which schools or districts are most effective in producing educational outcomes and 
it also produces a mix of schools that are much more representative of ah of the schools in the 
state. This latter point is especially important if the purpose of the exercise is to derive resource 
standards that could be applied to schools statewide. 

The BTO schools in this study are not selected on the basis of having the highest overall test 
scores in the state, but on the basis of substantially outperforming other schools statewide with 
like student populations. In addition, for comparison purposes, we used the same approach to 
also identify schools that are producing educational results that are much lower than expected 
given the students they enroll. We refer to these as low-performing (LP) schools. 
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A major premise underlying all approaches to education adequacy is that success is directly 
linked to the resources available to them. Under the successful schools approach, the premise is 
that the resources available to successful schools are the best indication of the resources needed 
by all schools to be successful. But what evidence is there that resource quantities constitute the 
primary distinguishing factor between successful schools and all others? Thus, the first question 
is whether BTO schools have more resources than LP or other schools. The second question is 
whether BTO schools are using their resources in more efficient ways than other schools in the 
state. 

To further explore this premise, we compare the total quantity and mix of resources found in 
these two sets of schools and against all other schools statewide. In addition, we compare the 
resource allocation practices across all three sets of schools (i.e., BTO, LP, and all others). How 
do these schools compare in regard to the total resources available to them, and in regard to the 
kinds of resources they employ? We also attempt to relate resource allocation practices to 
differences in student academic achievement by analyzing whether observable differences in 
resource levels are statistically related to academic performance. 

Of course, our ability to empirically measure these differences depends on the extent to which 
those aspects of BTO schools allowing them to outperform their peers are measurable and 
reflected in currently available data. For instance, if what distinguishes BTO from LP and all 
other schools is something other than measurable resource differences, to what extent is it 
possible to identify those factors? Do we observe different instructional, leadership, or 
professional development strategies in these schools? 

In an attempt to address these questions, we interviewed a subset of BTO and LP schools in an 
attempt to understand their resource allocation practices in relation to other factors they believe 
affect their student outcomes. The number of schools interviewed disproportionately favors BTO 
schools (n=19), but we also interviewed four LP schools to gain some comparative perspective. 
The purpose of these interviews was to understand what the leaders of these BTO and LP schools 
believed had most affected their ability to produce the academic results realized by their schools 
over the past several years. In addition to understanding their perspective on the relative 
importance of resources in regard to the performance of their school, we also wanted to know 
exactly what they believed were the most important factors affecting their students’ performance. 

It is not the intent of this study to estimate an “adequate” level of spending for all schools in 
California based on these BTO schools, but rather to shed some light on whether successful 
schools use their available resources in more efficient ways than other schools in California, and 
the extent to which the total amount or mix of resources or their unique use appears to explain 
their exceptional academic achievement levels. If the analyses show, and the interviews 
corroborate, that observed differences in student performance are primarily the result of other 
factors, are there implications for the use of the successful schools approach in further adequacy 
deliberations and for the overall consideration of educational adequacy in California? For 
example, how do these finding relate to the long-standing adequacy debate of more resources 
versus a more efficient use of the resources already available? 
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In summary, a major focus of this study is to explore how the successful schools approach has 
been used to measure educational adequacy. This question is largely addressed in Chapter 2, 
which reviews and summarizes findings from prior successful schools studies. This chapter also 
discusses some of the strengths and weaknesses of these alternative applications. 

Chapter 3 describes the data and methods used to address the subsequent research questions for 
this study. In summary, we used a mixed-method approach to examine school academic 
performance, resource allocation decisions, and other factors appearing to have led to school 
success. Statewide performance and demographic data were used as a basis for selecting beating- 
the-odds and low-performing schools, and school-level personnel profiles and distriet-level 
financial data were used to investigate schools’ resource allocation practices. Qualitative data 
methods (telephone interviews) helped identify issues not captured through the quantitative 
approaches, and to uncover strategies and other factors that led to or hindered academie success 
in selected schools. 

Chapter 4 presents methods for appropriately conceptualizing “successful schools” for the 
purposes of this study, and presents analyses of these schools. It presents analyses of whether 
resource differences are observed in these schools as compared to “low-performing” and other 
sehools statewide. It also attempts to assess whether there is any evidence that successful schools 
use their resources in a more efficient way than other schools (i.e., the extent to whieh academie 
performance can be predicted by the levels of resources in these schools and the types of students 
they serve). 

Chapter 5 presents the results of phone interviews with a subset of BTO and LP school principals 
in an attempt to assess the extent the resource findings from Chapter 4 are corroborated and to 
identify other factors said to be related to the educational outcomes produced by these schools. 
The final chapter of this report contains a discussion of possible implications for considering 
education adequacy in California. 
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Chapter II. Lessons Learned from Previous 
Research 



Introduction 

This chapter contains three sections reviewing prior research. The first section addresses the 
study’s first two research questions regarding how the successful schools approach has been used 
to consider educational adequacy, and the strengths and weaknesses of these alternative 
applications. 

The seeond section of this chapter is a literature review of studies that have analyzed elements of 
school effectiveness. In partieular, we will eoncentrate on studies that have examined high- 
performing schools serving low-income and minority populations, and what we have learned 
from them regarding their instructional strategies and praetices. This section provides the 
literature on which we based our theoretical framework of school effectiveness. This framework 
helped us develop the telephone interview protocols and suggested common characteristics and 
themes to look for aeross the schools we interviewed. 

In the last section, we will summarize studies that have focused on resource allocation practices 
at the school and district level. This seetion includes a brief literature review on resource 
allocation research, as well as produetion funetions in education. It provides a foundation for the 
theoretical model developed to analyze the resouree allocation practices of publie schools in 
California, as shown in Chapter 4. 
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Section I: Successful Schools Approach 

The basic idea behind the successful schools or districts approach is to identify schools or 
districts that are meeting certain performance benchmarks, and then identify their expenditure 
levels. This approach has been used primarily by John Augenblick and John Myers, although 
other researchers have used it as well. Exhibit 2.1.1 provides a list of the states in which the 
successful schools/districts approach has been used. 

Exhibit 2.1.1. Successfui Schoois/Districts Approach around the Nation 



Mississippi 


Augenblick, Van de Waters, and Myers (1993) 


Ohio. 


Augenblick (1997) 


New Hampshire 


Augenblick, Myers, and Silverstein (1997) 


Illinois 


Augenblick and Myers, Inc. (2001a) 


Louisiana 


Augenblick and Myers, Inc. (2001) 


Maryland 


Augenblick and Myers, Inc. (2001b)* 


Kansas 


Augenblick, Myers, Silverstein, and Barkis (2002)* 


Missouri 


Augenblick and Myers, Inc. (2003b)* 


Tennessee 


Augenblick (2003a)* 


Colorado 


Augenblick and Myers, Inc. (2003b)* 


Vermont 


National Conference of State Legislatures (2004) 


New York 


Standard and Poor’s School Evaluation Service (2004) 


Washington 


Fermanich, Mangan, Odden, Pious, Gross & Rudo (2006) 



* The successful schools/districts methodology was performed In conjunction with the professional judgment method. 



Each study uses different criteria to determine which schools or districts are considered 
“successful.” Augenblick and Myers tend to choose the criteria for each study by asking the 
state’s department of education to supply the criteria, as defined in the current state legislation 
(e.g., Tennessee and Maryland) or by a group of legislators or education providers (e.g., the 
Legislative Education Planning Committee in the case of Kansas). Consequently, each study 
employs a separate methodology for identifying successful schools and districts. The major 
differences are the type of academic performance tests used as cut points to determine the 
percentage of students meeting a certain level of performance; the overarching idea remains the 
same. Exhibit 2.1.2 provides an example of a set of criteria that have been used to identify 
successful districts. 



Page 6 



American Institutes for Research 




Successful California Schools in the Context of Educational Adequacy 



Exhibit 2.1.2. Example of Criteria Used to Determine Successful Districts 



Standardized achievement test: 

■ Absolute criteria: Based on a single year of performance, districts are chosen if a certain percentage 
of students is above a certain proficiency level 

■ Relative criteria: Usually based on two years of performance (sometimes three); districts are chosen 
if they are on track to achieving the absolute criteria. In other words, if districts are meeting their 
growth targets 

■ Minimum participation rate in standardized test (e.g., 80 percent) 

Filters: 

■ Maximum dropout rate (e.g., 2 percent) 

■ Minimum graduation rate (e.g., 85 percent) 

■ Other filters: maximum proportion of Title I schools in need of improvement, a certain attendance 
rate, and a minimum percentage of high-need students (e.g., English learners, special education 
students) 

Spending efficiency criteria: 

■ Districts will not be considered if they appear to be inefficient in their spending levels — if the 
predicted spending level for a district is lower than the actual level. 

Source: Augenbiick and Meyers adequacy studies in Tennessee, Illinois, Colorado, and Kansas. Standards & Poor’s School 
Evaluation Service, adequacy study for New York. 



Most of the successful schools/districts studies have identified successful districts, rather than 
schools. In some cases, schools have been identified instead of districts due to the small number 
of districts in the state. This was the case in Maryland, which only has 24 districts. In this study, 
the research team asked the state to determine which schools should be made the center of 
analysis. 

After the successful districts have been identified, it is then possible to estimate a base cost 
figure based on the districts considered successful. For the most part, this has been done in the 
following way: 

1. Examination of the basic expenditures and revenues of identified districts. Basic 
expenditures exclude spending for capital purposes, transportation, special education, 
English learner programs, and programs and services for at-risk pupils, as well as any 
adjustments for district characteristics — such as size or regional cost differences. 

2. Calculate a base cost figure using the basic expenditure figures of successful districts 
and, possibly, using a set of screening procedures to exclude some districts that, even 
though successful, might be considered to be unusual or inappropriate for some reason 
(based on demographic characteristics, for example). 

Even though analysts have suggested that the adequate expenditure level should be equal to the 
weighted average of the expenditures of all of the districts meeting the performance benchmark 
(as Augenbiick and Meyers recommend), in some of these studies the researchers have raised the 
possibility of using the average of only the bottom half of that sample, or even using the value of 
just the lowest-expenditure district in the sample. They describe this type of an adjustment as an 
“efficiency factor.” The rationale given for this type of adjustment is that just because districts or 
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schools are able to reaeh a given education target, does not mean that they are efficient in doing 
so. As there is generally more than one district reaching a given target, some of these districts are 
spending more and some less. If they are all reaching the speeified level of achievement, the 
reasoning is that those at the upper half of the spending speetrum must be less efficient in 
relation to those districts able to meet the target for less. Of course, this also drives down the 
value of the base adequate expenditure level and thus the state cost. An efficiency factor of this 
type was included in the suecessful schools studies conducted in New York, New Hampshire, 
Mississippi, and Maryland. 



Strengths of the Approach 

One appeal of this approach to defining education adequacy may be that the concept is easy to 
understand and appears logical. What could be a better source of information for the resources 
needed to produce a given set of educational results than the resources found in districts or 
schools already achieving them? In addition, this type of study is fairly straightforward to 
conduct, can be completed at a relatively low cost and within a short time span. The approach’s 
major strengths, discussed by Chambers and Levin (2006), are that it appears to contain a direct 
link between education costs and desired outeomes and the fact that the successful schools 
approach does not require resource cost data. 

A perceived strength from the perspective of those responsible for finding the funds for publie 
education (i.e., those from the legislative and administrative branehes of government) may be 
that it tends to produce lower estimates of the eost of educational adequacy (Baker, 2006). The 
philosophical underpinnings of this approach (i.e., that it may not be a matter of more money 
being needed but just the better use of existing money) also may have political appeal. 

One successful sehools report describes the approach as follows: “The suecessful sehool district 
approach is based on the simple premise that any district should be able to be as successful at 
meeting a set of objectives as those districts that actually meet those objectives provided that 
every district has the same level of funding that has been available to the sueeessful districts and 
that differences in student characteristics have been taken into consideration. . ..The strengths of 
the approach are that it is based on actual evidence that districts can be successful at a certain 
resource level and that the ways that resources are used can vary among successful districts; a 
weakness of the approach is that it makes no adjustments to the base cost to reflect 
uncontrollable cost pressures, since the eharacteristics of some districts might differ from those 
that have been successful.” (Augenblick & Myers, 2003; Colorado). 

Another report says this about the approach: “The successful schools approach considers all 
schools or districts in the state, identifies the ones that are meeting specified outeomes, and then 
treats the amount that those schools are spending as an adequate education funding level. The 
logic of this approach is simple and, with proper adjustments, proponents believe it is the best 
method for arriving at a cost of an adequate education.” (National Conference of State 
Legislatures, 2004; Vermont). 
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Limitations of the Approach 

Of course, proponents of all the approaehes think theirs is the best. The major limitation of the 
sueeessful schools is the “with proper adjustments” caveat mentioned above. This approaeh has 
no good way of dealing with one of the primary challenges of any adequacy analysis — how to 
adjust an overall adequaey estimate to meet the needs of varying pereentages of special needs 
students. Harr et al. (in progress) inelude a more detailed analysis of how various BTO 
applications have dealt with special student populations, and note that these studies have often 
been eonducted with other approaches to adequaey (e.g., professional judgment), whieh have 
provided adjustments to base sueeessful sehools adequacy estimates. The souree of some of the 
reeommendations in these eombined-methods approaches also raise questions. For example. 
Baker (2006:21) notes that there are instances wherein “arbitrary recommendations for marginal 
cost adjustments are attached to sueeessful sehools estimates after the fact.” 

In short, the major shorteoming of this approach is the diffieulty of attempting to define 
sueeessful schools in ways that are overly simplistie and whieh largely ignore the substantial 
impact of student eharaeteristics and special needs on the resourees needed to reach specified 
levels of educational outcomes. As articulated by Hanushek (2005), most schools that are 
underprivileged have not aehieved the standards that we use for these studies, and it is not 
possible to extrapolate what they should be spending to achieve those standards. For instance, if 
the schools under consideration have not yet achieved 95 pereent profieiency (which is the 
standard for NCLB), it is not appropriate to extrapolate the cost of achieving 95 pereent 
proficiency. 

Even with a more balaneed approaeh to seleeting sueeessful schools that fully incorporates 
student characteristics (e.g., the BTO analysis presented in Chapter 4 of this report), it is unclear 
how the resource findings from these sueeessful schools can be extrapolated to others. Even 
when the study does control for these types of school characteristics, the successful sehools 
model does not allow marginal analyses of spending on speeial populations, such as English 
learners, students with disabilities, or at-risk students. 

Another disadvantage, although eonsidered a strength by some researchers, is the assumption 
that edueation costs and desired outcomes are direetly linked. However, the faetors that make the 
sehools successful might be unobservable to the researcher, as in the ease of good leadership or 
more effeetive teaehers. Therefore, the sueeess of the school should not be automatieally 
attributed to the difference in what eaeh sehool spends. 

Section II: Selected Literature on Elements of School 
Effectiveness 

The literature presented in this section was used as the basis for the theoretieal framework of 
sehool effeetiveness used to develop the interview protocols, as well as suggested common 
characteristics and themes to look for across the sehools that were interviewed. 

Since the emergence of the effeetive sehools movement launehed by Ron Edmonds more than 
two deeades ago (Edmonds, 1982), the focal point for effective school researchers has been to 
identify schools successful in educating all students regardless of their socioeeonomie status. 
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Once identified, the task remains to recognize the common characteristics among those effective 
schools. Collectively, the body of research on effective schools during this period highlights the 
following school characteristics as correlated with high performance: 

1) Strong instructional leadership (Davis & Thomas, 1989; Edmonds, 1979; Purkey & 

Smith, 1983; Terry, 1996) 

2) Frequent monitoring of student progress (Levine & Lezotte, 1990; Purkey & Smith, 

1983; Newmann & Associates, 1996) 

3) Shared goals and professional community (Davis & Thomas, 1989; Darling-Hammond, 
1996) 

4) Parental involvement (Fullan, 1991; Levine & Lezotte, 1990; Purkey & Smith, 1983) 

5) Positive and academically focused school climate (Hoy and Hannum, 1997, Edmonds, 
1979; Rosenholtz, 1985) 

In the section that follows, we consider more-recent research on school effectiveness in order to 
provide a summary of the most current thinking regarding characteristics found in high 
performing schools that are educating high need populations. 

Williams, T., Kirst, M., Haertel, E., et al. (2005) selected a large sample of 257 elementary 
schools serving largely low-income students, as measured by the School Characteristics Index 
(SCI) — schools fell between the 25* to 35* percentile band on the SCI in 2003-04. The sample 
included high-, medium-, and low-performing schools within that band, as measure by the API. 
Surveys designed to explore school qualities, policies, and practices were sent to 257 principals 
and 5,500 teachers in the state. 

The survey included 350-400 items that were grouped into seven general domains. Answers of 
the different school types (i.e., high-, medium-, and low-performing schools) were compared 
using statistical methods. The study found that four specific domains were highly correlated with 
higher school API scores: I) prioritizing student achievement; 2) implementing a coherent, 
standards-based instructional program; 3) using assessment data to improve student achievement 
and instruction; and 4) ensuring availability of instructional resources. The survey also included 
questions related to parental involvement, teacher collaboration and development, and the 
enforcement of high expectations for student behavior, but the correlations between those 
domains and API scores, although positive, were not statistically significant. 

Parrish, T.B., Merickel, A., Perez, M., Linquanti, R., Socias, M., Spain, A., et al. (2006) 
administered telephone interviews to school administrators of beating-the-odds schools with high 
concentrations of English learners, as well as district administrators of five high-performing 
districts. The sample included 66 schools that showed high levels of academic performance with 
English learners. A comprehensive list, including 24 different elements, was analyzed and 
grouped into eight different domains. The factors that were most cited among school 
administrators as contributing to their success were: 1) staff capacity to address EL needs; 2) a 
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schoolwide focus on English Language Development; 3) shared priorities and expectations in 
regard to educating ELs; and 4) systematic, ongoing assessment and data-driven decision- 
making. District administrators also discussed strategies to support EL academic achievement, 
such as sustained, on-site technical assistance and professional development; strategic resource 
allocations, and timely provision and careful use of data. 

Springboard Schools is conducting the California Best Practices study (Oberman, Arbeit, 

Praglin, & Goldstein, 2005), a three-year study sponsored by the National Center for Educational 
Accountability (NCEA) that identifies successful high-poverty schools. Their ultimate goal is to 
investigate what these schools are doing right. During their first year (2004) the study focused on 
successful elementary schools; high schools were the focus of analysis in their second year 
(2005). In 2006 middle schools are being analyzed. In order to select high-performing sites for 
site visits, they looked at the California Standard Test (CST) student performance for three years, 
at enrollment in courses identified by the CDE as challenging courses, and at the percentage of 
students meeting the A-G subject requirement in the case of high schools. Comparison sites were 
selected among the schools in the “average” performance category as measured by the CST. 

They found: 1) successful schools use a consistent curriculum and frequent diagnostic tests; 2) 
teachers and administrators in suceessful schools seek out and use strategies that have been 
proven to be effective; and 3) successful schools invest in improvement. Regarding the last 
finding, they found that these schools invest their resources differently: schools hire coaches to 
help teachers and administrators, schools provide teachers with opportunities to collaborate and 
coach each other, and school districts invest in data systems. 

Using the Standards and Indicators for School Improvement (SISI), Kannapel & Clements 
(2005) conducted comprehensive site visits in eight high-performing, high-poverty schools in 
Kentucky. The results were compared with low-performing schools previously visited. The SISI 
categorizes school activities into nine standards; when site visit results were compared with those 
of low-performing schools, the eight study schools scored significantly higher on: 1) review and 
alignment of curriculum; 2) individual student assessment and instruction tailored to individual 
student needs; 3) caring, nurturing environment of high expectations for students, 4) ongoing 
professional development for staff that was connected to student achievement data; and 5) 
efficient use of resources and instructional time. 

The Center for Public Education synthesized relevant studies published within the last seven 
years on high-performing and high-poverty school around the nation and found ten factors that 
are consistently identified as characteristics of these schools.^ In their analysis those factors were 
separated into building blocks and five practices. The practices can be summarized as: 1) 
increased instructional time; 2) ongoing, diagnostic assessment; 3) parents as partners in 
learning; 4) professional development to improve student achievement; and 5) collaboration 
among teachers and staff. 



* To prepare this synthesis, the Center for Public Education started with an initial list of 300 studies. Seventeen 
studies were selected on the basis of their relevancy and evaluation rigor. 
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These reeent studies have found that sueeessful sehools share a eommon set of characteristics 
that are similar to those suggested by the body of effective schools literature that has been 
developed over the last two decades. The most-cited characteristics in these recent studies are: 

1 . A coherent and aligned curriculum 

2. Systematic use of performance data 

3. Efficient use of resources 

4. Environment of high expectations for students 

It is important to keep the limitations of these studies in mind, because they also pertain to our 
own findings for successful schools in Chapter 4. The practices identified by the research are 
associated with high academic performance, but a causal relationship cannot be assumed. These 
characteristics are not a list or “recipe” that will ensure high academic performance for schools, 
but rather should be considered part of a system in which particular strategies depend on others. 
In addition, the degree to which these practices are in place at a particular school is also a 
variable that should be considered. It is more common than not to find average or low- 
performing schools with these strategies in place; it is the way high-performing schools 
implement these strategies that seems to matter. The differences between the way average- 
performing and high-performing schools implement successful strategies are the intensity, the 
coherence, and the focus and willingness to stay focused over time (Oberman et al, 2005). 

Section III: Selected Literature on Resource Allocation 
Practices 

The literature on resource allocation in education is fairly limited. Not because it is seen as an 
uninteresting topic, but because researchers cannot agree on a more fundamental issue: the 
relevance of resources in determining student academic achievement. If the link between 
resources (or inputs) and education outcomes is not clear, it makes little sense to go one step 
further and ask for the optimum usage of resources such that academic achievement is 
maximized given a certain budgetary constraint. The research that studies this broader question 
of the impact of education inputs on outputs is referred to as education production function 
research. The literature in this area is extensive, and is not in agreement. Examples of the 
production function literature are found in Hanushek (1986), Burtless et al. (1996), Hanushek 
(1997), Krueger (1999), and Angrist and Lavy (2001). One of the most cited education 
production function studies is Eric Hanushek’ s (1997) meta- analysis that summarizes findings of 
277 studies of the effects of the teacher-pupil ratio on academic achievement, and 163 studies of 
the effects of expenditure per pupils on the same outcome measure. The analysis found that 72 
percent of the 277 studies show no statistically significant effect, while an additional 13 percent 
indicate a negative effect on achievement. The empirical evidence of the effect of expenditure 
per pupil is not very promising either. A total of 66 percent of the 163 studies don’t show a 
statistically significant effect, while an additional 7 percent indicate a negative relationship with 
student academic achievement. Similar results were found for teacher test scores (as a proxy of 
teacher quality), administrative inputs, and facilities. 

Of those studies that have tried to address the education resource allocation question, despite the 
well-known and valid concerns about education production function estimates, Lawton (1973) 
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focuses on the distribution of instructional resources in Detroit. He identifies three major factors 
as relevant in the production function of academic achievement: a) teacher experience, b) teacher 
academic training, and c) class size. The first two can be interpreted as a proxy for teacher 
quality, while class size is associated with the intensity at which a given teacher quality is 
applied to a group of students. His study indicates that instructional resources were not 
distributed evenly across schools in Detroit at the beginning of the 1970s. African-American 
students were taught, on average, by less experienced and less educated teachers than white 
students. Nevertheless, these African-American students had, on average, smaller class sizes than 
white students. His analysis shows that this last factor more than offset the teacher quality issue 
in the case of elementary schools, generating a positive correlation between percent of African- 
American students at the school and average instructional expenditure per student. 

Another resource allocation analysis is the one performed by Margaret Simms (1977) in the 36 
elementary schools in San Jose Unified School District. In this study, the main research question 
is how per pupil expenditure in teacher salaries varies by percentage of Spanish speaking 
students and total enrollment at the school. The findings suggest that teacher salary expenditures 
do not vary significantly with the percentage of Spanish speaking students, but that school 
enrollment is consistently correlated with this variable. Small schools have, on average, higher 
per pupil teacher salary expenditures than larger ones. The author links this relationship to class 
size, given that smaller schools also had smaller class sizes, which increased the per pupil 
expenditure figures. This study also analyzed the distribution of teaching experience and teacher 
education across schools with different percentages of minority students. The study found a 
negative correlation between the percentage of minority students and teacher experience and 
education. As the author states, “The pattern of teacher location was a combination of teacher 
preferences and district policy. Tenured teachers could request transfers out of “undesirable” 
schools into other ones. The pattern of transfer had been away from high-minority schools.” 

Another study by Hanushek, Rivkin, and Taylor (1996) uses a two stage estimation framework to 
analyze the impact of aggregation on schooling coefficients. It found that problems of omitted 
variables bias tend to increase along with the level of aggregation, causing analyses that use 
more aggregated data to overestimate systematically the influence of school expenditure related 
characteristics on student achievement. They add that these findings are consistent with the 
notion that increases in school expenditure used to reduce the teacher-pupil ratio and raise 
teacher salaries have had little systematic impact on student attainment, arguing that policies 
aimed at altering the incentive structure, and, thus, the ways in which resources are used appear 
much more likely to succeed than policies aimed simply at adding more resources to schools. 

Another study focuses on resource allocation in five “high-performing” schools. The study by 
Karen Hawley Miles and Linda Darling-Hammond (1998) analyzes important resource 
allocation changes introduced in five schools that have shown a strong improvement in student 
achievement despite serving challenging student populations. The authors attribute these results 
to the following changes in resource allocation practices: 

■ Reduction of specialized programs. “Pull-out” programs are costly and segregate 
students. Resources used in these types of programs should be brought back into the 
regular classroom and benefit all students. 
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■ More flexible student grouping. Students would benefit from multi-age grouping 
practices, in which they stay together for long periods of time. This promotes peer-to- 
peer learning. 

■ Structures that create more personalized environments. This may take the form of 
individual tutoring in reading or math, or teachers that serve as advisors to individual 
students or groups of students over long periods of time. 

■ Longer and varied blocks of instructional time. This policy is mainly targeted to high 
schools, in which students spend considerable amounts of time moving from one 
classroom to the next, for what the authors consider relatively small blocks of 
instructional time. The proposed solution would involve reducing the number of classes 
by increasing the length of blocks of instruction. 

■ More common planning time for staff. This allows teachers to learn from each other, 
sharing their instructional practices and fostering a collaborative working environment. 

■ Creative definition of staff roles. In order to reduce the amount of resources spent on 
administrative staff, and concentrate them on teachers, it is necessary that teachers extend 
their roles in schools. Teachers, for instance, may also act as advisors to a certain number 
of students each year, avoiding hiring guidance counselors or other administrative staff. 

A study by Betts, Rueben and Danenberg (2000) analyzed the distribution of school resources 
(measured by CBEDS data from 1999), and how that distribution is related to student academic 
achievement in California. The authors concluded that schools with the highest needs had fewer 
teaching resources, as measured by teacher education, experience, and credentials, and the 
availability of advanced placement courses. The authors also found that the differences in 
socioeconomic backgrounds explained most of the achievement gap. 
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Introduction 

This chapter describes the data sources and research methodologies used in the study. The first 
section addresses the process of identifying beating-the-odds (BTO) and low-performing (LP) 
schools in California. In the seeond section we describe the data sourees and methodologies that 
were used to: 1) analyze the level and the characteristies of resources used by BTO, LP and other 
public schools in their operations; 2) determine the level of per pupil spending in BTO and LP 
schools; 3) analyze what level of academic performance is predicted for BTO and LP schools by 
their level of resourees and students served; and 4) examine the extent to which BTO and LP 
schools are efficient in their resource use. We conelude the chapter with a description of how 
schools were selected for the telephone interviews, the process for condueting the interviews, 
and the type of data that was collected. 

The chapter’s sections are organized as follows: 

■ Section I: Identification of Beating-the-Odds and Low-Performing Schools. This 
section summarizes the data sources as well as the methodology used for seleeting 
BTO and low-performing schools. These schools are the subject of analysis in our 
exploration of resource allocation. In addition, a sample of these schools was chosen 
to conduct the telephone interviews — a procedure that is explained in Section III. 

■ Section II: Methodologies for the Analysis of Resource Allocation. This section 
presents the data sources that were used to eonduct the resource allocation analyses in 
BTO, low-performing, and other public schools in California as well as all the 
methodologies used in our exploration and analyses of resouree use in BTO, LP and 
other public schools. 

■ Section III: Telephone Interviews. This section describes the methodology for 
selecting the schools that were interviewed by telephone. It also presents the final 
sample of schools (with the corresponding BTO school names) as well as the type of 
data collected during those interviews. 



American Institutes for Research 



Page 15 




Successful California Schools in the Context of Educational Adequacy 



Section I: Identification of Beating-the-Odds and Low- 
Performing Schools 

Data Sources 

student and Demographic Characteristics 

Several databases were combined to create a profile for all public elementary and secondary 
schools in California. The California Basic Educational Data System (CBEDS) provided 
information on schools’ enrollment by grade and ethnicity, grade span, and school type 
(elementary, middle or high). These data were combined with variables from the Academic 
Performance Index (API), including the number of special education students, student mobility, 
graduation rates, class size, whether the school was a multi-track year-round school, parental 
education of the students, and the percentage of teachers that had regular or emergency 
credentials. Calworks provided the number of students that received or were eligible for free and 
reduced price meals, and the Language Census provided the number of English learners in each 
school, along with the primary language of each English learner. All of these variables were 
collected for each year from 2002 to 2005. 

Outcome Measures 

The main data source that was used to measure academic performance is the California 
Standardized Testing and Reporting (STAR) Program. This program started in October 1997 and 
all school districts were required to administer the tests to students in grades two through eleven. 
Today, the STAR program includes the results of a norm-referenced test (SAT-9 prior to 2002, 
and CAT/6 afterwards), and the California Standard Tests (CSTs). Student-level test scores are 
not publicly available; therefore available grade-level test scores for each school were used to 
assess the performance of California schools from 2002 to 2005. These results are available at an 
aggregate level (i.e., for each grade 2 to 1 1) for all students in the school that are taking the test, 
as well as for subgroups when 10 or more students have valid test scores. In addition to using 
the overall grade-level test scores, we included outcome measures for the following subgroups: 
English learners, students eligible for free and reduced price lunch, Hispanic students, and 
African-American students. 

The primary outcome measure that was used to select beating-the-odds elementary and middle 
schools is the CST English language arts (ELA) and mathematics scale scores from 2002 to 
2005. All elementary and middle school students take the same CST ELA and mathematics tests 
corresponding to their grade level. However, high school students take the math test 
corresponding to the math course they are enrolled in (for example, geometry or algebra). As a 
result, there is no consistent CST mathematics test that can be compared between all high 
schools. Given that limitation, we decided not to use the CST mathematics results in the 
selection of beating-the-odds high schools. Instead, we used the California High School Exit 



^ STAR also includes the California Alternate Performance Assessment (CAPA) and the Spanish Assessment of 
Basic Education (SABE/2). They are administered to students with significant cognitive disabilities and Spanish- 
speaking English learners who were enrolled in public schools in California less than 12 months, respectively. 

^ STAR available test scores can be downloaded from the following California Department of Education website: 
http://star.cde.ca.gov/ . 
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Exam (CAHSEE)"^ tests in English language arts and mathematics from 2002 to 2005, in addition 
to the CSX ELA results for high schools. All high school students are required to take the 
CAHSEE ELA and mathematics tests for the first time in tenth grade; therefore, we used the 
passing rates of tenth graders as the outcome indicator for high schools. Some criticize the use of 
the CAHSEE because it tests very basic skills, but we decided that it was most important to have 
a consistent comparison of the math achievement between schools. 

There are several reasons underlying our decision to use the CSTs and CAHSEE instead of the 
Academic Performance Index (API) as the outcome measures. The first is that we wanted to be 
able to use an outcome measure for English learners at the school, and EL scores are not 
disaggregated in the years we were analyzing. Second, given that the API is a composite of 
different measures it encompasses more variability than one single test. In addition, the API does 
not remain consistent over time. That is, it is a moving target because the academic subjects 
included in the calculation of the API, as well as the weights that are applied to these subjects, 
change over time, which introduces variability or noise when used across years. However, it 
should be noted that the CSX and CAHSEE results are major components of the API results. 



Methodology for Selection 

High-Performing Schools 

Given the instability of test scores over time, the schools in this study were selected using several 
years of data. Eor elementary and middle schools, four years of test scores were used. Eor high 
schools, we used three years of data. As an indicator for academic achievement in our analysis, 
we used CSX ELA and mathematics scale scores.^ In order to generate a school-level average 
academic achievement indicator from the CSX scale scores, we first standardized the average 
CST score of each grade using the corresponding statewide CST score distribution. This step is 
necessary because CST scores are not vertically equated, which makes scale scores non- 
comparable across grades. Standardized grade averages were then weighted by the relative 
enrollment in each grade to obtain a school-level average academic performance measure. This 
was done for all students, English learners (EEs), Hispanic students, African-American students, 
and students eligible for free and reduced price lunch (ERPL) from 2002 to 2005. OLS 
regressions were then used to identify beating-the-odds and low-performing schools. 



Beginning in 2005-06, students must pass the CAHSEE to earn a high school diploma. In addition, this test is used 
to calculate the Academic Performance Index (API) for state accountability purposes and Adequate Yearly Progress 
(AYP) to meet federal No Child Left Behind requirements. 

^ In the case of high schools, we used CAHSEE ELA and mathematics test results in addition to the CST ELA 
results. 
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Using each of the above as outcome variables, we estimated a panel regression for each year, and 
for each school type separately (elementary, middle, and high schools). The following is the 
regression that was estimated: 



Y\ = p^ + P‘PiX\ + s\ (1) 

where Y is the outcome variable in subject t (either ELA or mathematics) for student category p 
(all students, students that receive free or reduced price lunch, English learners, Hispanic 
students, or African-American students) for school i in year y. X is a vector of school 
characteristics that includes the percentage of students that receive free or reduced price lunch 
(ERPL), the percentage of EEs in the school, the percentage of EEs that speak Spanish, the 
percentage of students with disabilities, the percentage of parents that did not complete high 
school, the percentage of parents that have some college education, the percentage of parents that 
have a college degree, and the percentage of parents that have a graduate degree. Eor high 
schools, parental education was not included in the regressions due to missing values. 

Once this model was estimated, the fitted value (Ytpiy) was calculated for each category of 
student p in school i for year y and the residual (stpiy) was saved. Using the residuals, schools 
were then identified as beating the odds (BTO) schools in each of the five student categories, p, 
in each year y. Exhibit 3.1.1 lists the requirements a school must meet in order to be identified as 
BTO in each of the five student categories. 

After charter and magnet schools were removed from the sample, there were 61 elementary, 7 
middle, and 35 high schools identified as BTO schools in all five categories and in all four 
years.*’ 



^ There were no high schools that were heating the odds in all four years, so we limited the criteria to heating the 
odds in three years (2003 to 2005) for high schools only. 
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Exhibit 3.1.1. Criteria for Seiecting Beating-the-Odds Schoois 



Elementary and Middle Schools 

For all students: 

■ CST ELA and mathematics residual for all students is more than .75 standard deviations 
above the mean residual for students in all elementary or middle schools in California. 

For subgroups: FRPL, EL, Hispanic, and African-American students: 

■ Residual CST ELA and mathematics for these subgroups of students is more than .75 of a 
standard deviation above the mean residual for these subgroups of students in all 
elementary or middle schools in California. 

■ Residual CST ELA and mathematics can be missing. The school may not have that 
subpopulation of students (or an insufficient number of students to be considered 
accountable for that subgroup), and still may be part of the pool of schools that are beating 
the odds. 

High Schools 

For all students: 

■ CST ELA residual and CAFISEE ELA and mathematics residuals percent passing rate is 
positive. 

For subgroups: FRPL, EL, Hispanic, and African-American students: 

■ Residual CST ELA for all subgroups is positive and residual CAHSEE ELA and 
mathematics percent passing rate for all subgroups is positive. 

■ Residual CST ELA and residuals CAHSEE ELA and mathematics percent passing can be 
missing. The school may not have that subpopulation of students (or an insufficient number 
of students to be considered accountable for that subgroup), and still may be part of the 
pool of schools that are beating the odds. 



Low-Performing Schools 

Low-performing schools were then identified as schools that have a negative error term (stpiy) in 
all four years and all student categories (i.e., all students, ELs, Hispanic students, African- 
American students, and students that receive free or reduced price lunch). This means that these 
schools have been performing at a lower level than would be expected given the student 
population they serve. The total number of low-performing schools selected is 76 elementary 
schools, 32 middle schools, and 5 high schools. 
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Selected Schools 

High-performing schools were selected if they were beating the odds every year during a four- 
year period (three years for high schools). The same was true for low-performing schools. The 
following exhibit shows the number of schools that were beating the odds each and every year 
from 2002 to 2005. As it shows, there is a lot of instability in the test score results. Schools could 
outperform one year, and the next perform as expected or underperform. For example, 365 
elementary schools were beating the odds during 2002; from 2003 to 2005 about 280-300 
elementary schools were also beating the odds. However, only 61 elementary schools were 
beating the odds every year during that time period. Similar numbers are seen for middle and 
high schools. 



Exhibit 3.1.2. Number of BTO Schoois Each Year, 2002-2005 





2002 


2003 


2004 


2005 


All years 


Elementary 


365 


307 


292 


280 


61 


Middle 


41 


39 


37 


38 


7 


High Schools 


N/A 


82 


150 


190 


35 


Total 


406 


428 


479 


508 


103 



One of the striking results shown in this table is the fact that there aren’t many beating-the-odds 
schools in the state. The fact that a total of only about 100 schools in California were actually 
performing higher than expected during this time period is a result that should be of concern. 
This study focuses on what BTO schools are doing that has allowed them to achieve this high 
level of performance. However, it is beyond the scope of this work to investigate why there are 
few schools in the state that have reached this high level of performance given the population 
they serve. 
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The following exhibit shows the distribution of 2005 CST ELA residuals for all schools, with 
BTO schools in black, low-performing schools in dark gray, and the remaining public schools in 
light gray. It shows that BTO schools do not seem to be outliers. Or in other words, there seem to 
be schools that are beating the odds but not highlighted as BTO schools. This is because this 
graph only presents the residuals of the CST for 2005. Some schools could be beating the odds 
this year (2005) but they not consistently outperforming their peers during the previous years. 



Exhibit 3.1.3. Distribution of Residuals across Poverty Level for BTO Schools and Low- 
Performing Schools 
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One way to more accurately graphically represent BTO schools is to graph the average residual 
during the four- year period rather than focusing on just one year and one subject. The following 
exhibit presents the average residual across all years and subjects for elementary schools. Even 
though this better represents BTO schools, there are still some schools that for a given poverty 
level have a higher average ELA residual than some BTO schools. One reason could be that 
those schools outperform in only three of the four years under analysis, but still have an average 
residual during the four-year period that is higher than some BTO schools. 

Exhibit 3.1.4. Distribution of the Average Residual across Poverty Level for BTO Schools and Low- 
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We also constructed a success rate indicator to show BTO and LP schools more accurately. This 
success rate goes from 0 to 1, with 1 meaning that the school meets all the requirements to be 
considered a BTO school. Zero means that the school does not meet any of the requirements. For 
example, a school that met the requirements to be considered beating the odds during two years, 
but missed the requirements for the subpopulations during the other years would have a success 
rate close to 0.5. Exhibit 3.1.5 shows success rate distribution for elementary schools across 
poverty levels. As shown, BTO schools have a success rate of 1, and LP schools have a success 
rate of 0. 

Exhibit 3.1.5. Success Rate Distribution across Poverty Levei for BTO and Low-Performing 
Schoois 




One interesting result that can be observed in all of these graphs of BTO and LP schools is the 
fact that LP schools are clustered among high poverty levels. This result means that there aren’t 
any low-poverty schools that consistently perform at a lower level than what is expected for 
them. Low-performing schools in California tend to be schools with high concentrations of poor 
students. Though we found high-performing schools across the entire spectrum of poverty levels, 
high-performing schools were more likely to be at the lower poverty levels. More about the 
demographic characteristics of BTO and LP schools will be presented in the following chapter. 
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Section II: Methodologies for the Analysis of Resource 
Allocation 

In this section we describe the data and methodologies for each of the three major analyses that 
we conducted in exploring resource allocation in BTO, low-performing, and public schools in 
general:^ 



■ Section II-A describes the process of building a statewide school-level personnel 
database that was used as a cornerstone for a comprehensive descriptive analysis of 
personnel resources in California schools. 

■ Section II-B describes the methodology and data used to determine the level of 
spending for BTO and LP schools. 

■ Section II-C develops a theoretical model for conceptualizing the problem of 
resource allocation when schools attempt to maximize student outcomes under 
budgetary constraints. This model is then used for measuring how resources and 
student characteristics predict the level of academic performance in BTO and LP 
schools. In addition, this model is used to analyze how efficient BTO and LP schools 
are in their use of teacher resources. 

Although these methods are distinct, and approach resource allocation from different angles, they 
build off of the same data sources and are complementary. Exhibit 3.2.1 provides an overview of 
the extant data informing our study of resource allocation and highlights some of the key 
variables that are used. 



^ A similar theoretical framework and methodological approach was used for a study of charter schools conducted 
hy AIR that is part of the “Getting Down to Facts” project. 
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Exhibit 3.2.1. Data Used for Analyses of Resource Allocations (CBEDS 2004-05) 


CBEDS - 2004/05 


PAIF - Certificated Staff 


Position/Assignment/Course 


Administrators 


FTEs in administrative positions (e.g., principals, vice principals, 
department chairs, program administration) 


Teachers 


FTEs in teaching positions (e.g., self contained classroom teachers, core 
subject teachers, instructional support specialists) 


Pupil Support Staff 


FTEs in pupil support positions (e.g., counselors, librarians) 


Characteristics 


Education 


Highest Educational level achieved (i.e., doctorate, master's, bachelor's, 
less than bachelor's) 


Experience 


Years of educational service in total and in district. 


Teaching Credentials 


FTEs designated as having full teaching credentials. 


Status 


FTEs designated as long term subs, probationary, tenured or other. 


SIF - Classified Staff 


Position 



Number of full-time and part-time staff designated as teaching assistants, 

Paraprofessional teacher aides, pupil service aides, and library aides. 

Number of full-time and part-time staff designated as providing clerical or 



Office/ Clerical 


administrative support. 


Other 


Number of full-time and part-time staff designated as managers, 
custodians, food service staff, bus drivers, noon duty supervisors, and 
staff below the level of assistant, deputy, or associate superintendent. 


Characteristics 


None Available 


Public School Databases 


- School Level Indicators 



Types of public schools include Special Education School, County 
Community School, Youth Authority Facility, Opportunity School, Juvenile 
Court School, Other County-Wide Programs, Elementary School, Single 
Elementary School in District, Intermediate/Middle School, Alternative 
School, Junior High School, K-12 School, High School, Single High 
School in District, Continuation High School, Community Day School, 

School Type State Special School, and Adult Education Center. 

Charter Identifies charter schools 

Enrollment Grade by grade enrollment 

Grade Span 

SACS - 2004/05 

LEA Revenue and Expenditure Report 

District Level Expenditures 

Certified Salaries 

Non - Certified Salaries 

Employee Benefits 

Books and Supplies 

Services and Other 

Operating Expenses 

Capital Outlay 

STAR - 2004/05 

CST 

English Language Arts and 

Mathematics Scale Scores All students in grades 2-11 
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Section ll-A: School-Level Resource Database 

In order to explore how personnel resources differ between BTO and low-performing schools, 
we built a comprehensive school-level personnel database, providing a profile of the quantities 
and characteristics of staff in California’s public schools. 

Data Sources and Methodologies 

o 

Using 2004-05 CBEDS data derived from the Personnel Assignment Information Form (PAIF), 
we obtained information on certificated personnel in California. The PAIF files accounts for all 
certificated staff in the state, their major functions (assignments) in a given school or district, and 
the relative amount of time spent on a particular assignment. We aggregated this information to 
the school and district level across three broad personnel categories: administrators, teachers, and 
pupil support staff. These personnel categories are aligned with administrative employees, 
teachers, and pupil services employees as defined in Education Code 41401 and referenced in the 
Administrative Manual for CBEDS Coordinators and School Principals, October 2005? 

Although district-level employees are reported in PAIF, it is not possible to assess how district 
staffs are allocated across the schools within each district. We also captured more detailed 
assignment categories (nested within the administrator, teacher and pupil support categories) and 
aligned them with the subject code groupings in CBEDS. Quantities of personnel within each 
assignment category are reported in PAIF in terms of full time equivalents (FTEs). PAIF also 
provides information on personnel characteristics, such as gender, experience in education, 
highest degree level obtained, teaching status, and whether or not they are fully credentialed. In 
addition, PAIF contains information on the courses that schools offer, such as course enrollments 
and whether or not the course satisfies the University of California graduation requirement. 

Using CBEDS School Information Form (SIF)^*^ for 2004 - 05, we obtained information on the 

number of part-time and full-time classified staff at each school site." As SIF data are not 

collected in a manner that allows FTE reporting, in our analyses we assume part time status as 

0.5 FTE and full time status as 1 FTE. Information on experience and education is not available 

for classified staff. School level characteristics, such as school type, grade span, and enrollment 

12 

were obtained from the Public Schools database. 

The Standardized Account Code Structure (SACS) and the Charter School Alternative Form 
Database of 2004-05 provide annual revenue and expenditure figures for all school districts in 
California, as well as for charter schools that reported their financial data independently from 
their districts. Expenditures are disaggregated into certified salaries, non-certified salaries, 
employee benefits, books and supplies, services and other operating expenses, capital outlay, and 
other outgo. Teachers' salaries are a subcategory within certified salaries. 



* PAIF files downloaded from: http://www.cde.ca.gov/ds/ss/cb/filespaif.asp. 

® Please refer to Appendix 1 for a more detailed description of the staff that are included in PAIF as well as a more 
detailed description of the specific personnel assignments that are included within each personnel category. 

SIF Sections A, D and E downloaded from: http://dq.cde.ca.gov/DataQuest/downloads/sifade.asp. 

A classified employee is an employee of a school district in a position not requiring certification. The numbers of 
classified staff do not include preschool, adult education, or Regional Occupation Program classified employees. 
Public Schools database downloaded from: http://www.cde.ca.gov/ds/si/ds/pubschls.asp. 
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As an indicator for academic achievement for all public schools in California, we used the same 
outcome measure that was used for the selection of beating-the-odds and low-performing 
schools: CST ELA and mathematics scores standardized across grades. However, for this 
analysis only the year 2004-05 was included. 

Using all of these data sources, we constructed a comprehensive database that includes: 

■ Total certified and non-certified personnel at the school and district level 

■ Characteristics of the personnel (e.g., education level, number of years of experience, 
tenure status) 

■ Class sizes and caseload 

■ Total salaries for different personnel staff 

■ Expenditures for different categories (e.g., books and supplies, capital outlay, and other 
outgo) 



Section ll-B: Methodology for Costing Out BTO and LP Schools 

This section describes the methodology and data used to cost out the total expenditure that BTO 
and low-performing schools incur in their operations. 

Teacher and administrator wage equations play a crucial role in this exercise because spending 
on personnel salaries and benefits represents about 81 percent of district expenditures. These 
equations are both defined as follows: 



W =a^ +a^ edu+a 2 exp -i- exp -i- rwi + £ 

where the dependent variable is an average teacher (or administrator) salary (that does not 
include benefits), which depends on educational attainment, a non-linear function of the teaching 
experience, and a relative wage index (RWI). This index is based on the average teacher salary 
(for a given level of education and experience) in 30 different California regions. We then 
divided the average teacher salary of each region, by the Los Angeles Unified School District 
(LAUSD) average teacher salary (i.e., this region was used as the base, or in other words, the rwi 
is 1 for LAUSD). 

We estimate this wage equation for teachers and administrators of all public schools in California 
using the comprehensive database described in the previous section. In particular, we use salary 
information and personnel characteristics such as educational level and experience. 

Once the weighted least squares wage equations for teachers and administrators were 
estimated, we combine the coefficients with the school-level average educational attainment 



Source: Ed-Data, Expenditures by Category (2004-05). http://www.ed-data.kl2.ca.us . 

The average teacher salaries for the 30 regions were obtained from “Getting Down to Facts” research by Heather 
Rose. 

The number of teachers and administrators within districts was used as the weight. 
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and experience of teachers and administrators. Then, we multiply these school-level average 
wages by the total full time equivalents (FTEs) of teachers and administrators at the school to 
obtain the total spending in teacher and administrator salaries, respectively. 

In order to complete the picture of personnel expenditures, it is also necessary to consider what 
schools spend on pupil support and non-certified personnel. Unfortunately, it was not possible to 
estimate reliable wage equations for these types of staff because of data constraints. So, we 
simply use state-level average salaries for these types of personnel ($53,529 and $51,730 for 
pupil support and non-certified staff, respectively). We then multiply these estimates by the total 
number (FTEs) of support staff (e.g., counselors, psychologist, librarians), and non-certified staff 
(e.g., paraprofessionals) in each school to obtain the total spending in pupil support and non- 
certified staff, respectively. 

We then obtained the total spending on personnel salaries by summing up the expenditures on 
teachers, administrators, pupil support staff, and non-certified personnel wages. Benefit ratios 
were estimated dividing the total spending in salaries by the total spending in employee benefits 
in each district (using SACS). This district-level benefit ratio was then applied to the total 
expenditures on personnel salaries. These estimations produced what we call in this report total 
personnel per-pupil spending for each beating-the-odds and low-performing school. 

Then, using SACS, we obtained at the district level: 1) per pupil central office administration 
spending (this includes board and superintendent spending, other general administration, and 
centralized data processing); 2) per pupil non-personnel spending (including textbooks and core 
curricula materials and supplies); 3) per pupil plant expenditures (plant maintenance and 
operations, facilities acquisitions and construction, facilities rents and leases); and 4) per pupil 
spending on other services (this includes travel and conferences, memberships, professional 
consulting services). 

Finally, we added total personnel per-pupil spending to all district-level per-pupil spending (i.e., 
central office administration, non-personnel, plant expenditures, and other services) to estimate 
the total per pupil spending in California. The results of this methodology are presented in 
Chapter 4. 



Section ll-C: Resource Allocation as an Optimization Problem 

In this section we describe the theoretical model that was used in our analysis of how much of 
the academic performance level in BTO and LP schools is explained by their level of resources 
and student characteristics. This model was also used in our study of how efficient BTO and LP 
schools are in their use of resources. We define a simplified model that attempts to simulate the 
general goal and constraints that schools face. The model is a simplification of the context in 
which public schools operate — it leaves out complex and relevant factors including teacher 
unions, long- term labor contracts, the role of the district, and parental involvement. 
Nevertheless, we see this as a relevant exercise that should help inform our discussion about the 
efficient use of available resources in a world of budgetary constraints. 
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The model is specified as follows. An educational institution that operates in a competitive 
environment has the following objective function: 

Max L= Y enr (1) 

where Y is an outcome measure of average school-level student academic achievement. The 
variable enr represents the enrollment at the school. The production function equation is given 
by the following formula: 



(X R t^Ct c* 

Y = A edu exp ( Y (povY (elsY 

enr 



( 2 ) 



Where edu and exp represent the average years of education and teaching experience of the 
teachers at the school, respectively, and tea indicates the total number of teachers working at the 
school. The terms pov and els represent the percent poverty and percent English learners at the 
school, respectively. a,p,y,d,rj and A are parameters. This academic production function states 
that average student achievement depends on teacher and student characteristics, as well as class 
size. 



Replacing Equation 2 in Equation 1, we get the following objective function: 



— —a p tea 



-s —n 



Max L = A edu exp ( Y pov els enr 

enr 



( 3 ) 



The school faces several constraints in this optimization process. These constraints are given by 
the following equations: 



Total Costs =C + H enr -I- W tea (4) 

W=0Q+d^edu+02Qfop + 6^rwi + £ (5) 

Equation 4 defines the cost structure of the school. C represents fixed costs (e.g., facility costs). 
The school also faces variable costs that depend on student enrollment. H may be associated, for 
instance, with costs related to special education services that the school faces per student 
enrolled. Einally, we also need to consider labor costs. These are equal to the product of the 
average annual salary (W ) times the number of teachers working at the school (tea). 

Equation 5 states that the average annual cost of labor depends on the characteristics of the 
teachers hired by the school. We expect 0^ and 02 to be positive parameters. The term rwi 
represents a relative wage index that takes geographical as well as urban-rural wage differences 
into account. 

The results and the analyses performed with this model are presented in Chapter 4. 
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Section III: Telephone Interviews 

In addition to conducting a comprehensive analysis of resource allocations in beating-the-odds 
schools, low-performing schools, and statewide schools; we performed telephone interviews with 
administrators of a sample of these schools. 



Selected Sample of Schools 

Given that we wanted to focus on high-need high-performing schools for the telephone 
interviews, it was necessary to narrow down the number of BTO schools to conduct the 
telephone interviews. From the original sample of 103 beating-the-odds schools; schools that did 
not meet the school-wide 2004-2005 API growth target or did not meet all 2005 AYP criteria 
were removed from the list (a total of 1 1 schools). Once these schools were removed, the final 
selection of schools was hand-picked by the research team. Variables that were considered when 
making the final selection were parental education, percentage of students receiving free or 
reduced price lunch, and racial diversity of the school. We also verified that schools were not 
selective with the student population they enroll (i.e., schools were not pre-testing or selecting 
students for enrollment, and in addition, students were not dismissed due to low academic 
performance). The primary goal was to select schools that are serving students with high needs 
(e.g., high percent poverty, high percent of English learners, low parental education). The final 
sample of BTO schools that were contacted for a telephone interview included 16 elementary 
schools, 3 middle schools, and 5 high schools. 

In order to select “comparison” schools, or in other words, schools that have a similar mix of 
student characteristics compared to BTO but are performing at a lower level, a one-to-one 
matching technique was used for selection among the sample of low-performing schools. Once 
matched schools were identified, the research team selected a small sample of 3 elementary, 2 
middle, and 3 high schools for conducting telephone interviews. It is important to mention that 
the comparison schools were not selected from among the schools with the highest poverty levels 
because we did not want to select schools whose low performance level was mainly explained by 
their high-poverty status. 

As a way of displaying the overall performance of the BTO and comparison schools. Exhibit 
3.3.1 shows the distribution of 2005 CST ELA residuals for all schools. Notice that this exhibit 
displays only an overall school-level performance in a single year (2005), therefore those 
highlighted as BTO schools are not necessary those performing at the highest level in the graph. 
BTO schools, in order to be selected as such, needed to be performing above their expected level 
for the past four years. In addition, all subgroups in the school needed to be outperforming as 
well, not just the overall school. The following exhibit displays BTO schools in black, low- 
performing schools in dark gray, and the rest of public schools in light gray. 



The CST English language arts fitted values were used for matching low-performing with beating-the-odds 
schools. 
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Exhibit 3.3.1. Distribution of CST ELA Residuals, with Interviewed Schoois Highlighted, 2005 




School Names 

The final sample of interviewed beating-the-odds schools, or schools that have been performing 

17 

for the past four years at high levels given the student population they serve, is the following: 

■ Monte Vista Street Elementary, Los Angeles Unified 

■ Fremont Primary, Calipatria Unified 

■ Azusa High, Azusa Unified 

■ Cahuenga Elementary, Los Angeles Unified 

■ Cantara Street Elementary, Los Angeles Unified 

■ Clifford Street Elementary, Los Angeles Unified 

■ Commonwealth Avenue Elementary, Los Angeles Unified 

■ Mayall Street Elementary, Los Angeles Unified 

■ Normont Elementary, Los Angeles Unified 

■ Leroy L. Doig Intermediate, Garden Grove Unified 

■ George R. Moscone Elementary, San Francisco Unified 

■ Dewey Elementary, San Diego Unified 

■ Olivewood, National Elementary 

■ Baldy View Elementary, Upland Unified 

■ Ramona High, Riverside High 

■ Garden Grove Elementary, Los Angeles Unified 

■ Orange High, Orange Unified 

■ North Salinas High, Salinas Union High 

Due to confidentiality agreements, the names of the “comparison” schools are not included. 
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Telephone Interview Process and Response Rate 

Once BTO and comparison schools were selected, the research team solicited these schools to 
participate in telephone interviews. The first step was obtaining district permission to contact the 
school principals. All 16 districts agreed that we could contact the principals, although one 
district indicated that two of the BTO schools that we had selected were not suitable for our 
study. These two middle schools were dropped and not replaced. The next step was calling the 
principal to solicit participation and schedule the interviews. Five elementary schools refused to 
participate in the study (four BTO and one comparison school) and were replaced with similar 
schools. An additional eight schools were unresponsive to our phone calls and were not replaced 
due to time constraints. One BTO telephone interview was scheduled for early August, but in the 
end the principal could not participate, not allowing enough time for replacement. Exhibit 3.3.2 
presents the total number of schools that were contacted for interviews, the number of interviews 
that were conducted, and the number of schools that were dropped from the study and not 
replaced. 



Exhibit 3.3.2. Selected, Interviewed and Dropped Schools for Telephone Interviews 





BeatIng-the-Odds Schools 


Comparison Schools 






Elementary 


Middle 


High 


Elementary 


Middle 


High 


Total 


Total selected schools 


16 


3 


5 


3 


2 


3 


32 


Total Interviewed 
schools 


13 


1 


4 


3 


2 


0 


23 


Dropped without 
being replaced 


3 


2 


1 


0 


0 


3 


8 


Response rate 


87.5% 


33.3% 


80.0% 


100.0%o 


100.0% 


0.0% 


75.0% 



Data Collected through Telephone Interviews 

The one-hour telephone interviews included 25 questions that covered the following topics (the 
protocols used for the telephone interviews can be found in Appendix 2): 

■ Background information about the school and the principal 

■ School strategies for success (including probes — when relevant — about curriculum, 
instruction, professional development, teacher collaboration, use of data, district 
support, and parental involvement) 

■ Challenges to increasing academic achievement 

■ School leadership 

■ Funding 

■ Hiring and firing practices 

■ Additional staff 

■ Professional development 

■ District support 

■ Feasibility of reaching an API of 800 
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These same topics were covered for both BTO and comparison schools, although there were 
slight differences in the way in which the questions were phrased. For example, BTO schools 
were asked to describe the primary strategy implemented that has influenced their high results, 
while comparison schools were asked to describe the strategies implemented that have affected 
the performance of their school. Another variation in the interviews was that schools with an API 
below 800 were asked about the feasibility of reaching that goal and what kind of changes 
(including resources) would be required to get closer to an 800. Schools with an API above 800 
were asked whether they had sufficient resources to continue at that level. 
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Resuits 



Introduction 

A major objective of this study is to determine if there are resource differences between groups 
of schools showing varying degrees of success in regard to student performance. In this chapter, 
we focus predominantly on the allocation of human resources, for three important reasons. First, 
personnel are the most necessary resource to provide educational services to all students in the 
traditional public school setting. Second, as mentioned in the previous chapter, districts spend 
approximately 81 percent of their total expenditures on personnel salaries and benefits. Third, 
statewide data are most readily available for personnel resource analyses. 

This chapter is divided into three main sections: First we present a descriptive analysis of the 
types of students that are served in schools with different levels of performance, we then 
consider how these schools are staffed and the extent to which the characteristics of their 
personnel differ (e.g., education and experience levels). We proceed by assessing resource 
differences with more care by controlling for important student demographics. Second, we 
estimate the level of total spending at these schools and compare those estimates with actual 
spending levels in the state. The third section goes one step further by analyzing how much of 
the academic achievement gap between high-performing schools and other public schools we are 
able to explain through differences in resources and the types of students they serve. We repeat 
this analysis to explain the academic achievement gap that exists between low-performing 
schools and other public schools. In addition, this section analyzes school efficiency by 
developing a framework that measures whether beating-the-odds (BTO) schools use their teacher 
resources more efficiently than low-performing (LP) schools. 

It is also important to keep the limitations of these analyses in mind; given the nature of the 
variables available from extant statewide data, we are forced to approach resource allocation 
indirectly, using variables or combinations of variables as proxies for resource allocation 
practices. For instance, we cannot capture organizational features of schools that shape how 
resources are used, such as incentives or programs put in place to influence the dynamics 
between personnel and interactions with students, nor can we examine key structural features 
such as the scheduling, sequencing, and length of classes. And other important school 
characteristics pertaining to leadership, turnover and the kind and frequency of professional 
development teachers receive, for example, are not routinely documented in statewide databases. 
The lack of such data makes a comprehensive analysis of resource allocation more difficult. 

This chapter is organized into three sections: 

• Section I: Results of the Descriptive Analysis of Personnel Resources in BTO and LP 
Schools. This section presents the results of a descriptive analysis as well as statistical 
analyses that aim to describe differences in the distribution and characteristics of personnel 
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resources in beating-the-odds (BTO), low-performing (LP), and other public schools in 
California. 

• Section II: Results of Costing Out BTO and LP Schools. This section presents the results 
of a costing-out exercise that estimates the total educational spending level of beating-the- 
odds and low-performing schools. These estimates are also compared with actual statewide 
per pupil spending. 

• Section III: Resource Allocation as an Optimization Problem. This section presents the 
results of estimating a theoretical model that helps understand 1) how much of the 
achievement gap that exists between BTO and other public schools, as well as between LP 
and other public schools, is due to differences in their resources and student characteristics, 
and 2) how BTO and LP schools compare in efficiency in the allocation of their teacher 
resources. 

Section I: Results of the Descriptive Analysis of Personnel 
Resources in BTO and LP Schools 

As detailed in the methodological chapter (Chapter 3), using CBEDS as the primary data source, 
we built a comprehensive school-level personnel database to explore how personnel resources 
differ between beating-the-odds, low-performing schools, and other public schools in the state. 
The first part of this section presents a descriptive analysis of how personnel are allocated among 
these schools and which are the characteristics of their staff. In the second part we conduct the 
same analysis while controlling for demographic characteristics. 



Describing Personnei and Resource Aiiocation with CBEDS Data 

Describing personnel with CBEDS data allows us to consider whether or not BTO schools are 
staffed differently (in terms of the numbers and characteristics of their employees) in comparison 
to other public schools. Because our personnel database accounts for each employee in each 
school in California, addressing this question is straightforward. We defined a series of variables 
describing resource levels that capture information on the quantities of certified and classified 
staff and average levels of education and experience for teachers and administrators. 

We also specified a series of variables describing resource ratios. Examples of these variables are 
teacher experience-to-education level, and pupil support staff-to-teachers. In some cases these 
variables were defined to broadly approximate dimensions of school organization, such as 
capacity to support teachers (pupil support-to-teachers) and support to principals (vice principals 
and clerks-to-principals). In other cases, the variables simply describe the distribution of staff 
across assignments (e.g., percentage of teachers assigned to core subjects versus electives), 
seniority, and credentialing status. We recognize that the interpretation of some of these ratios 
may not be straightforward; however they are presented to help understand the results of our 
efficiency analysis presented in Section III of this chapter. 

In addition, given that elementary schools are structurally different from middle and high 
schools, and may have different needs and priorities, the analyses in this chapter are presented 
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separately for elementary schools, whereas middle and high schools are grouped together. For 
comparability, we exclude charter, alternative, continuation, special education, state special, 
juvenile hall, community day, and adult education schools. In our analyses, we make 
comparisons between three groups of schools: beating-the-odds schools, low-performing 
schools, and other traditional public schools in California (refer to Chapter 3 for a description of 
how BTO and low-performing schools were selected). 

Exhibit 4.1.1 provides demographic characteristics and school sizes for BTO, LP, and other 
public schools in the state. Overall, the most striking thing to note is the relative similarity 
between BTO and all other schools in relation to the LP schools. Compared to the large category 
of all other schools, the LP schools have approximately 50 percent more students in poverty 
(77.2 percent versus 50.9 percent), about 60 percent more ELs (39.1 percent versus 25.4 
percent), and a substantially higher percentage of minority students (about 50 percent higher). LP 
schools also are substantially larger than BTO schools across all grade levels. 



Exhibit 4.1.1. Characteristics of Schooi Groups 

Beating-the-Odds (BTO) Schools 




Elementary 


Middle 


High 1 


1 Total 




(N 


= 60) 


(N 


= 7) 


(N = 


= 34) 


(N 


= 101) 






Standard 




Standard 




Standard 




Standard 




Average 


Deviation 


Average 


Deviation 


Average 


Deviation 


Average 


Deviation 




(%) 


(%) 


(%) 


(%) 


(%) 


(%) 


(%) 


(%) 


% Poverty 


57.9 


33.0 


51.3 


27.5 


54.2 


13.1 


56.2 


27.4 


% ELs 


33.7 


23.9 


23.0 


26.2 


22.5 


8.7 


29.2 


20.8 


% Hispanic 


49.4 


34.1 


45.4 


32.1 


63.4 


14.7 


53.9 


29.5 


% African-American 


5.3 


7.9 


3.8 


5.0 


12.4 


13.0 


7.6 


10.3 


Enroilment 


552 


264 


663 


224 


2,656 


774 


1,268 


1,109 


Low-Performing (LP) Schools 




Elementary 


Middle 


High 1 


1 Total 




(N 


= 76) 


(N 


= 32) 


(N 


= 5) 


(N 


= 113) 






Standard 




Standard 




Standard 




Standard 




Average 


Deviation 


Average 


Deviation 


Average 


Deviation 


Average 


Deviation 




(%) 


(%) 


(%) 


(%) 


(%) 


(%) 


(%) 


(%) 


% Poverty 


81.0 


18.7 


71.0 


21.5 


60.2 


9.1 


77.2 


20.0 


% ELs 


42.4 


15.4 


33.2 


16.3 


28.0 


12.2 


39.1 


16.1 


% Hispanic 


54.8 


18.3 


54.5 


24.5 


54.1 


22.7 


54.7 


20.2 


% African-American 


21.9 


10.3 


24.1 


18.6 


33.3 


17.9 


23.0 


13.6 


Enroilment 


777 


311 


1,545 


835 


3,252 


1,184 


1,104 


801 


Other Public Schools 




Elementary 


Middle 


High 


Total 




(N = 


5,200 ) 


(N = 


1,178) 


(N = 


934) 


(N = 


= 7,312) 






Standard 




Standard 




Standard 




Standard 




Average 


Deviation 


Average 


Deviation 


Average 


Deviation 


Average 


Deviation 




(%) 


(%) 


(%) 


(%) 


(%) 


(%) 


(%) 


(%) 


% Poverty 


54.0 


30.4 


49.3 


27.5 


35.7 


24.5 


50.9 


29.9 


% ELs 


28.7 


22.8 


19.5 


16.3 


14.0 


13.8 


25.4 


21.6 


% Hispanic 


44.6 


30.2 


43.0 


28.1 


36.2 


25.9 


43.3 


29.5 


% African-American 


7.1 


11.0 


7.6 


10.8 


7.3 


10.5 


7.2 


10.9 


Enrollment 


570 


269 


940 


473 


1,714 


1,077 


776 


618 
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Exhibit 4.1.2 displays the number of BTO and LP schools in each poverty quartile; BTO schools 
are spread out fairly evenly across the quartiles, while the majority of LP schools are in the 
higher quartile of poverty. 



Exhibit 4.1.2. Schooi Counts by Statewide Poverty Quartiie 





BTQ Schools 


LP Schools 


Other Public Schoois 


Total 




(n) 


(%) 


(n) 


(%) 


(n) 


(%) 




Quartile 1 

(0 - 25% poverty) 


16 


16 


3 


3 


1,863 


25 


1,882 


Quartile 2 

(25 - 53% poverty) 


23 


23 


12 


11 


1,846 


25 


1,881 


Quartile 3 

(53 - 77% poverty) 


37 


37 


34 


30 


1,811 


25 


1,882 


Quartile 4 

(77- 100% poverty) 


25 


25 


64 


57 


1,792 


25 


1,881 


Total 


101 


100 


113 


100 


7,312 


100 


7,526 



Exhibits 4.1.3 (elementary schools) and 4.1.4 (middle/high schools) provide a personnel profile 
for the typical BTO, LP and other public schools. The accompanying discussion provides an 
overview of how certain resource levels and characteristics contrast between school groups. The 
regression analysis that follows this discussion considers detailed differences in these variables, 
while controlling by demographic characteristics across school groups. 

BTO and LP School Staffing 

Teachers 

With regards to teacher staffing in elementary schools, BTO schools have a slightly higher ratio 
of teachers to pupils compared with other public schools (5.19 versus 5.15 respectively; variable 
#2) while LP schools have the lowest ratio (5.01). The trend is different for middle and high 
schools, where other public schools have the highest ratio (4.86), followed by LP schools (4.34), 
and BTO schools have the least number of teachers relative to student enrollment (4.1 1). 

In addition, there are differences regarding the educational attainment of teachers in these three 
different school categories. In particular, there are slightly higher proportions of elementary 
teachers with a master’s or doctorate degree in BTO and other schools (29.7 percent and 29.5 
percent, respectively; variable #9) than in LP schools (20.2 percent). The same pattern is evident 
in middle and high schools: BTO and other schools have higher percentages of teachers with 
advanced degrees (35 percent and 33.7 percent respectively; variable #9) in comparison to LP 
schools (21.8 percent). 

Probably the most notable difference between LP, BTO, and other public schools in regard to 
teachers is the lower level of professional experience that teachers at LP schools possess. In 
elementary schools, average teacher experience is almost two years lower than in other schools 
(approximately 11 versus 13 years, variable #10). This difference is even bigger at the middle 
and high school level, with a teacher experience of 10.1 years in LP schools versus 12.23 and 
12.81 in BTO and other public schools, respectively. 
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Administrators 

There are also variations in the characteristics of administrators in these three different school 
types. Among elementary schools, the administrator-to-student ratio is higher in BTO schools 
(0.34, variable #3) than in LP and other schools (0.27 and 0.28, respectively). Meanwhile, there 
are more administrators, per 100 students in other middle and high schools than in the 
corresponding BTO and LP schools (0.30 versus 0.26 and 0.28, respectively; variable #3). 

Furthermore, a greater proportion of administrators in elementary BTO schools possess advanced 
degrees than their counterparts in LP and other schools (85.4 percent versus 71.9 percent and 
84.8 percent, respectively; variable #12). They also have more years of experience in the same 
district compared to LP and other public schools (18.13 versus 12.87 and 13.85, respectively; 
variable #14). In middle and high schools, these differences between administrators in BTO, LP, 
and other schools are less pronounced. 

Percentage of Staff in Teaching, Administrative, and Pupil Support Positions 
The typical elementary BTO school has a slightly greater proportion of its employees in teaching 
positions (92.7 percent versus 91.6 percent; variable #21) and administration (6.0 percent versus 
4.9 percent; variable #22), and a smaller proportion in pupil support roles (1.4 percent versus 3.5 
percent; variable #23), relative to LP schools. In addition, LP schools show the highest 
percentage of staff in pupil support assignments (e.g., counselors, psychologist), when compared 
with BTO and other public schools (3.5 percent versus 1.4 and 2.3 percent, respectively). 
However, when we look at middle and high schools, these differences are virtually nonexistent 
(see variables #20, 21, & 22). 
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Exhibit 4.1.3. Eiementary Schooi Personnei Resource Profiie 





Variable/ 










Other 




Resource 






BTO 


LP 


Public 




Category 


Var # 


Variable Description 


Schools 


Schools 


Schools 






1 


Certified staff per 100 pupils 


5.60 


5.50 


5.59 






2 


Teachers per 1 00 pupils 


5.19 


5.01 


5.15 






3 


Administrators per 100 pupils 


0.34 


0.27 


0.28 




Staff per 


4 


Pupil support staff per 1 00 pupils 


0.08 


0.21 


0.15 




pupil 


5 


Classified staff per 1 00 pupils 


2.87 


2.08 


2.86 






6 


Paraprofessionals per 100 pupils 


1.51 


0.90 


1.31 


CO 




7 


Clerical office staff per 100 pupils 


0.55 


0.47 


0.50 


Q) 

> 

CD 

_l 

CD 




8 


Other classified staff per 1 00 pupils 


0.82 


0.71 


1.06 




9 


Teachers: average total years of education*'*’ 


17.23 


17.12 


17.26 


D 






Teachers: percent with bachelor’s degree or less 


70.3% 


79.8% 


70.5% 


CO 

CD 






Teachers: percent with master’s or doctorate 


29.7% 


20.2% 


29.5% 


cn 




10 


Teachers: average total years of experience in education 


12.84 


11.00 


12.96 




Education & 


11 


Teachers: average total years of experience in district 


11.26 


9.24 


10.82 




experience 


12 


Administrators: average total years of education*'*’ 


18.24 


17.89 


18.34 








Administrators: percent with bachelor’s degree or less 


14.7% 


27.5% 


15.3% 








Administrators: percent with masters or doctorate 


85.4% 


71 .9% 


84.8% 






13 


Administrators: average total years of experience in education 


22.72 


17.47 


20.28 






14 


Administrators: average total years of experience in district 


18.13 


12.87 


13.85 






15 


Average class size: kindergarten self-contained classrooms 


19.49 


20.40 


21.58 




Class sizes 


16 


Average class size: grades 1 - 3 self-contained classrooms 


19.73 


19.82 


20.13 






17 


Average class size: grades 4 - 5 self-contained classrooms 


29.31 


29.76 


30.38 




Experience : 
education 


18 


Teachers: total years of experience per total years of education 


0.74 


0.64 


0.75 




19 


Administrators: total years of exp. per total years of education 


1.25 


0.97 


1.10 




20 


Pupil support staff: total years of exp. per total years of education 


0.87 


0.79 


0.81 




Distribution of 


21 


Share of total staff who are teachers 


92.7% 


91 .6% 


92.9% 


CO 


staff across 


22 












major 


Share of total staff who are administrators 


6.0% 


4.9% 


4.8% 


tr 

CD 

O 


assignments 


23 


Share of total staff who are pupil support staff 


1 .4% 


3.5% 


2.3% 




24 


Teachers per administrator 


17.08 


20.83 


21.80 


O 

CO 




25 


Teachers per pupil support staff 


19.90 


28.52 


29.49 


CD 

cn 


Support 

capacity 


26 


Administrators per pupil support staff 


0.94 


1.57 


1.39 




27 


Paraprofessionals per teacher 


0.28 


0.18 


0.24 




28 


Share of teaching staff providing mentoring support 


0.3% 


1 .5% 


1.1% 






29 


Share of teaching staff providing instructional support 


1 .8% 


2.7% 


2.7% 






30 


Administrators + clerical office staff per principal 


3.62 


4.75 


3.11 




Teacher status 


31 


Share of total teachers designated as probationary or temporary 


15.4% 


25.8% 


20.0% 




& 


32 


Share of total teachers with tenure 


52.0% 


62.1% 


71 .0% 




characteristics 


33 


Share of total teachers with full credentials 


95.3% 


93.3% 


96.6% 



Notes: 

(a) As CBEDS only includes the discrete education attainment level of staff, we create a continuous education variable in the 
following way: 



• Less than bachelor's degree: f 2 years of education. 

• Bachelor's degree: 16 years of education. 

• Bachelor's degree plus 30 or more semester hours: 17 years of education. 

• Master's degree: 1 8 years of education. 

• Master's degree plus 30 or more semester hours: 19 years of education. 

• Doctorate: 21 years of education. 
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Exhibit 4.1.4. Middle & High Schooi Personnei Resource Profiie 





Variable/ 

Resource 

Category 


Var # 


Variable Description 


BTO 

Schools 


LP 

Schools 


Other 

Public 

Schools 






1 


Certified staff per 100 pupils 


4.63 


4.88 


5.41 






2 


Teachers per 1 00 pupils 


4.11 


4.34 


4.86 






3 


Administrators per 100 pupils 


0.26 


0.28 


0.30 




Staff per 


4 


Pupil support staff per 1 00 pupils 


0.27 


0.27 


0.25 




pupil 


5 


Classified staff per 1 00 pupils 


2.14 


2.37 


2.27 






6 


Paraprofessionals per 100 pupils 


0.68 


0.88 


0.76 


CO 




7 


Clerical office staff per 100 pupils 


0.56 


0.63 


0.55 


Q) 

> 




8 


Other classified staff per 1 00 pupils 


0.89 


0.86 


0.96 


_l 

Q) 




9 


Teachers: average total years of education‘s’ 


17.23 


17.03 


17.31 


D 






Teachers: percent with bachelor’s degree or less 


65.0% 


78.2% 


66.3% 


CO 

Q) 






Teachers: percent with master’s or doctorate 


35.0% 


21 .8% 


33.7% 


cn 




10 


Teachers: average total years of experience in education 


12.23 


10.10 


12.81 




Education & 


11 


Teachers: average total years of experience in district 


10.36 


8.50 


10.24 




experience 


12 


Administrators: average total years of education’s’ 


18.33 


18.00 


18.21 








Administrators: percent with bachelor’s degree or less 


20.4% 


27.4% 


21 .9% 








Administrators: percent with masters or doctorate 


79.6% 


72.6% 


78.1% 






13 


Administrators: average total years of experience in education 


18.76 


16.16 


18.62 






14 


Administrators: average total years of experience in district 


14.12 


14.68 


12.54 




Class sizes 


18 


Average case load: core subjects 


153.75 


143.77 


142.31 






19 


Average case load: electives 


190.44 


211.27 


183.59 




Experience : 
education 


20 


Teachers: total years of experience per total years of education 


0.71 


0.59 


0.74 




21 


Administrators: total years of exp. per total years of education 


0.85 


0.82 


0.88 






22 


Pupil support staff: total years of exp. per total years of education 


1.02 


0.90 


1.02 




Distribution of 


23 


Share of total staff who are teachers 


88.6% 


88.8% 


89.7% 




major 


24 


Share of total staff who are administrators 


5.5% 


5.7% 


5.5% 


CO 


assignments 


25 


Share of total staff who are pupil support staff 


5.9% 


5.5% 


4.8% 


00 




26 


Teachers per administrator 


19.04 


17.11 


17.97 


cn 




27 


Teachers per pupil support staff 


15.89 


18.01 


20.89 


o 




28 


Administrators per pupil support staff 


0.99 


1.19 


1.28 


o 

CO 


capacity 


29 


Paraprofessionals per teacher 


0.17 


0.21 


0.16 


IT 




30 


Share of teaching staff providing mentoring support 


1 .3% 


0.8% 


1 .6% 






31 


Share of teaching staff providing instructional support 


0.1% 


0.2% 


0.4% 






32 


Administrators + clerical office staff per principal 


15.95 


13.00 


9.25 






34 


Share of Total Teachers who teach core subjects*’’ 


57.8% 


66.2% 


59.6% 




emphasis 


35 


Share of Total Teachers who teach elective subjects’"’ 


20.3% 


14.1% 


18.9% 




36 


Elective Teachers per Core Teachers 


0.36 


0.22 


0.33 




Teacher status 


37 


Share of total teachers designated as probationary or temporary 


20.0% 


28.5% 


27.0% 




& 


38 


Share of total teachers with tenure 


53.5% 


37.4% 


65.7% 




characteristics 


39 


Share of total teachers with full credentials 


88.9% 


78.3% 


92.2% 



Notes: 

(a) As CBEDS only includes the discrete education attainment level of staff, we create a continuous education variable in the 
following way: 



• Less than bachelor's degree: 12 years of education. 

• Bachelor's degree: 16 years of education. 

• Bachelor's degree plus 30 or more semester hours: 17 years of education. 

• Master's degree: 1 8 years of education. 

• Master's degree plus 30 or more semester hours: 1 9 years of education. 

• Doctorate: 21 years of education. 

(b) Core subjects include: Mathematics, English, History and Social Science. 

(c) Elective subjects include: Humanities, Arts, Music, Physical Education, Computer Education and Foreign Language 
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Seniority Status 

BTO elementary schools have lower percentages of teachers in tenure (52 percent versus 71 
percent; variable #32) and in probationary status (15.4 percent versus 20 percent; variable #31) 
when compared with other public schools. Meanwhile, LP elementary schools have higher 
proportions of probationary instructors (25.8 percent) and tenured teachers (62.1 percent) relative 
to BTO schools. Similar patterns exist in middle and high schools: 65.7 percent of teachers in 
public schools hold tenure, compared with 53.5 percent in BTO schools and 37.4 percent in LP 
schools (variable #38); while 27 percent of the teachers in public schools are probationary, 
compared to 20 percent and 28.5 percent in BTO and LP schools, respectively (variable #37). 

Teacher Credentials 

There are no salient differences in the percentage of teachers with full credentials in elementary 
schools. In middle and high schools, on the other hand, other public schools have the highest 
share of fully credentialed teachers (92.2 percent; variable #39), followed by BTO schools (88.9 
percent), and LP schools have the lowest proportion (78.3 percent). 

Summary 

In general, while we observed some differences in staffing between these three school categories, 
they do not appear to be substantial. The most important difference appears to be the level of 
experience that teachers and administrators hold: LP school teachers and administrators have less 
experience in education-related work. In addition, it is worthy of note that LP schools have more 
staff focused on providing pupil support (e.g., counselors), which may be a reflection of the 
higher needs of their students given their higher poverty level. 



Statistical Analyses of Resource Allocations 

Although the resource allocation descriptions presented above provide a valuable summary of 
the levels and characteristics of personnel among BTO, LP, and other public schools, with a 
simple average it is difficult to disentangle these differences in personnel and the degree to 
which they are tied to differences in the student populations they serve. For example, as 
displayed in Exhibit 4.1.1, LP schools tend to be significantly larger, have a higher level of 
poverty and higher percentages of English learner and African-American students than BTO 
schools. 

The previous section also showed that these schools differ in the characteristics of the teachers 
they employ. Exhibit 4.1.5 displays a box plot summarizing the distribution of the percentage of 
teachers holding master’s or doctoral degrees (y-axis), broken out by poverty quartile. Even 
though BTO schools have, on average, a higher percentage of teachers with advanced degrees 
across all quartiles, the variation across school types in quartiles 1,3, and 4 is much less than that 
seen in the overall results (Exhibit 4.1.3 and 4.1.4). 

Differences across schools in the qualifications of teachers have been largely documented (Betts, 
Rueben and Danenberg, 2000). Low performing schools and schools with a high percentage of 
students with high-needs (e.g., poverty level, ELs) tend to have teachers with low education and 
experience level. One plausible explanation is that this difference results from some 
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organizational feature of high-performing schools (or low-performing schools), such as the 
ability of the school leadership or district to hire and retain highly qualified staff. 

Exhibit 4.1.5. Box Plot for the Average Percentage of FTE Teachers Holding Master’s or Doctoral 
Degrees, by School Group and Poverty Quartiles 
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Regression Results 

To control for differences in student poverty between the three kinds of schools and to assess if 
the differences observed in the previous analysis are statistically significant, we used a 
multivariate OLS regression model to explore how personnel resources in BTO and LP schools 
differ in comparison to other public schools while controlling for student characteristics. This 
model does not imply a causal relationship between the dependent and independent variables. 
This model is designed to explore how personnel resources in BTO and LP schools differ in 
comparison to other public schools. The regression used is as follows: 

Y„ = po + + hLPi,+ ^3STUi,+ s,s 



1 8 

Where Y equals resource level or ratio i for school s, BTO is a dummy variable indicating 
whether a school is a beating-the-odds school, LP is a dummy variable indicating whether a 
school is a low-performing school and STU is a matrix with student demographic variables that 
include percent poverty, percent ELs, percent Hispanic, and percent African American. 

Exhibit 4.1.6 displays the regression results for elementary schools, and Exhibit 4.1.7 the results 
for middle and high schools combined. The models presented in these exhibits contain as control 
variables the percent poverty and percent ELs at each school. Below we discuss in detail 
differences in resource levels and characteristics of staff in BTO, LP and other public schools 
that are observed in the regression results. 

As our dependent variables, we use the resource variables defined in Exhibit 4.1.3 and Exhibit 4.1.4 (teachers per 
100 pupils, average years of education, and average years of experience, for example). 

Please refer to Appendix 3 for regression results including percent Hispanic and percent African American as 
additional control variables. 
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Exhibit 4.1.6. OLS Regression Resuits - Eiementary Schoois 

Beating-the- 







Var. 

# 




Odds 

Schools 

(BTO) 

(Coefficient) 

(A) 


Low- 

Performing 
Schools (LP) 
(Coefficient) 
(B) 


Percent 
Percent English 

Poverty Learners 

(Coefficient) (Coefficient) 
(C) (D) 


Constant 

(E) 


Observations 

(F) 


R-squared 

(G) 






1 


Certified staff per pupil 


0 


-0.003 


0.011*** 


-0.009*** 


0.053*** 


5,334 


0.006 






2 


Teachers per pupil 


0.001 


-0.003 


0.01*** 


-0.012*** 


0.049*** 


5,334 


0.020 






3 


Administrators per pupil 


0.001 


0 


0.001*** 


0 


0.002*** 


5,334 


0.008 




Staff per pupii 


4 


Pupil support staff per pupil 


-0.001 


0 


-0.001 


0.003*** 


0.001*** 


5,334 


0.002 


jn 




5 


Classified staff per pupil 


0.001 


-0.011 


0.020*** 


-0.016** 


0.022*** 


5,321 


0.003 


> 

0) 




6 


Paraprofessionals per pupil 


0.002 


-0.005 


0.008** 


-0.005 


0.01*** 


5,321 


0.001 


o 




7 


Clerical office staff per pupii 


0.001 


-0.001 


0.004*** 


-0.005*** 


0.004*** 


5,321 


0.034 


u 




8 


Other classified staff per pupil 


-0.002 


-0.005 


0.008*** 


-0.006* 


0.008*** 


5,321 


0.002 


o 

(0 




9 


Teachers: average total years of education 


-0.013 


-0.071 


-0.269*** 


0.045 


17.388*** 


5,335 


0.034 


QC 




10 


Teachers: average total years of experience in education 


0.168 


-1 .277*** 


-0.436* 


-4.101*** 


14.372*** 


5,335 


0.079 




Education & 


11 


Teachers: average total years of experience in district 


0.648 


-1.278*** 


0.468** 


-3.117*** 


1 1 .46*** 


5,335 


0.036 




experience 


12 


Administrators: average total years of education 


-0.097 


-0.375*** 


-0.212*** 


-0.171** 


18.509*** 


5,041 


0.014 






13 


Administrators: average total years of experience in education 


2.616** 


-1.912* 


-1 .732*** 


-3.434*** 


22.238*** 


5,041 


0.021 






14 


Administrators: average total years of experience in district 


4.516*** 


-1.422 


2.262*** 


-1.226 


12.984*** 


5,041 


0.007 






15 


Average class size: kindergarten self contained classrooms 


-2.386*** 


-0.453 


-3.98*** 


2.622*** 


22.957*** 


4,952 


0.023 




Ciass sizes 


16 


Average class size: grades 1 - 3 self contained classrooms 


-0.379 


-0.006 


-1 .428*** 


0.604*** 


20.73*** 


5,018 


0.026 






17 


Average class size: grades 4 - 5 self contained classrooms 


-0.816 


0.349 


-3.564*** 


-0.128 


32.342*** 


4,837 


0.007 






18 


Teachers: total years of experience per total years of education 


0.01 


-0.072*** 


-0.013 


-0.24*** 


0.826*** 


5,335 


0.076 






19 


Administrators: total years of exp. per total years of education 


0.155** 


-0.087 


-0.078** 


-0.183*** 


1.199*** 


5,041 


0.019 






20 


Pupil support staff: total years of exp. per total years of education 


0.044 


-0.017 


0.078 


-0.293*** 


0.854*** 


2,457 


0.009 




Distribution of 


21 


Share of total staff who are teachers 


-0.001 


-0.01* 


-0.014*** 


0.005 


0.935*** 


5,336 


0.006 


O 


staff across 






0.01** 


-0.004 


0.02*** 


-0.004 


0.039*** 


5,336 


0.030 


(0 


major 




Share of total staff who are administrators 
















QC 

0> 


assignments 


23 


Share of total staff who are pupil support staff 


-0.009* 


0.014*** 


-0.006** 


-0.001 


0.026*** 


5,336 


0.005 


U 




24 


Teachers per administrator 


“4.422*** 


0.315 


-6.77*** 


3.993*** 


24.305*** 


5,041 


0.034 


o 

<n 




25 


Teachers per pupil support staff 


-9.75 


1.933 


-13.969*** 


1 1 .024*** 


33.581*** 


2,457 


0.010 


o 

cc 




26 


Administrators per pupil support staff 


-0.415 


0.185 


-0.267* 


0.8*** 


1 .3*** 


2,457 


0.008 






27 


Paraprofessionals per teacher 


0.055 


-0.064** 


0.12*** 


-0.18*** 


0.223*** 


5,321 


0.012 






28 


Share of teaching staff providing mentoring support 


-0.009*** 


0 


0.006*** 


0.018*** 


0.002*** 


5,335 


0.048 






29 


Share of teaching staff providing instructional support 


-0.009 


0 


0.004 


-0.004 


0.026*** 


5,335 


0.001 






30 


Administrators + clerical office staff per principal 


0.373 


1 .087*** 


1.017*** 


2.365*** 


1 .844*** 


4,798 


0.160 




Teacher status 


31 


Share of total teachers designated as probationary or temporary 


-0.046** 


0.051** 


0.028** 


-0.002 


0.185*** 


5,335 


0.004 




& 


32 


Share of total teachers with tenure 


-0.185*** 


-0.036 


-0.155*** 


-0.082*** 


0.818*** 


5,335 


0.062 




characteristics 


33 


Share of total teachers with full credentials 


-0.011 


-0.02*** 


-0.036*** 


-0.025*** 


0.992*** 


5,335 


0.065 



* significant at 10%; ** significant at 5%; *** significant at 1% 
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Exhibit 4.1.7. OLS Regression Resuits - Middie & High Schoois 










Beating-the- 






















Odds 


Low- 




Percent 












Var. 




Schools 


Performing 


Percent 


English 












# 




(BTO) 


Schools (LP) 


Poverty 


Learners 
















(Coefficient) 


(Coefficient) 


(Coefficient) (Coefficient) 


Constant 


Cbservations 


R-squared 










(A) 


(B) 


(C) 


(D) 


(E) 


(F) 


(G) 






1 


Certified staff per pupil 


-0.007 


-0.002 


0.02*** 


-0.052*** 


0.054*** 


2,189 


0.013 






2 


Teachers per pupil 


-0.007 


-0.003 


0.018*** 


-0.048*** 


0.049*** 


2,189 


0.013 






3 


Administrators per pupil 


0 


0 


0.001** 


-0.003*** 


0.003*** 


2,189 


0.004 




Staff per pupii 


4 


Pupil support staff per pupil 


0 


0 


0 


-0.001 


0.003*** 


2,189 


0.001 


o 

> 

Qi 


5 


Classified staff per pupil 


-0.002 


0.000 


0.022*** 


-0.032*** 


0.018*** 


2,170 


0.031 




6 


Paraprofessionals per pupil 


-0.001 


0.001 


0.011*** 


-0.014*** 


0.005*** 


2,170 


0.014 


O 




7 


Clerical office staff per pupii 


0 


0.001 


0.003*** 


-0.005*** 


0.005*** 


2,170 


0.017 


U 




8 


Other classified staff per pupil 


-0.001 


-0.001 


0.009*** 


-0.012*** 


0.008*** 


2,170 


0.033 


o 

(0 




9 


Teachers: average total years of education 


-0.035 


-0.176*** 


-0.482*** 


0.138** 


17.498*** 


2,190 


0.103 


QC 




10 


Teachers: average total years of experience in education 


-0.039 


-1.807*** 


-1.618*** 


-3.146*** 


14.051*** 


2,190 


0.072 




Education & 


11 


Teachers: average total years of experience in district 


0.519 


-1 .244** 


-0.771** 


-1 .883*** 


10.892*** 


2,190 


0.027 




experience 


12 


Administrators: average total years of education 


0.119 


-0.155 


-0.162* 


-0.05 


18.284*** 


2,103 


0.006 






13 


Administrators: average total years of experience in education 


0.541 


-1.892* 


-1.51* 


-0.977 


19.433*** 


2,103 


0.009 






14 


Administrators: average total years of experience in district 


1.618 


1.585 


1.112 


1.653 


1 1 .776*** 


2,103 


0.009 




Ciass sizes 


15 


Average class size: core subjects 


12.123*** 


0.811 


-12.658*** 


25.757*** 


143.387*** 


2,165 


0.012 




16 


Average class size: electives 


5.678 


18.919** 


14.72** 


31.45*** 


171.891*** 


2,144 


0.031 




Experience : 
education 


17 


Teachers: total years of experience per total years of education 


0 


-0.098*** 


-0.073*** 


-0.191*** 


0.803*** 


2,190 


0.065 




18 


Administrators: total years of exp. per total years of education 


0.02 


-0.094 


-0.067 


-0.065 


1.061*** 


2,103 


0.007 




19 


Pupil support staff: total years of exp. per total years of education 


-0.021 


-0.046 


0.092* 


-0.241** 


0.88*** 


1,877 


0.004 




Dist. of staff 


20 


Share of total staff who are teachers 


-0.011* 


-0.009 


0.001 


-0.001 


0.896*** 


2,190 


0.002 




across major 


21 


Share of total staff who are administrators 


-0.002 


0 


0.012*** 


-0.01* 


0.052*** 


2,190 


0.008 


(0 


assignments 


22 


Share of total staff who are pupil support staff 


0.013*** 


0.009* 


-0.013*** 


0.01* 


0.052*** 


2,190 


0.011 


’■P 




23 


Teachers per administrator 


1.564 


-0.344 


-4.191*** 


3.876* 


19.101*** 


2,103 


0.007 


QC 




24 


Teachers per pupil support staff 


-5.272* 


-3.479 


2.61 


-0.92 


19.949*** 


1,877 


0.004 


U 


Support 

capacity 


25 


Administrators per pupil support staff 


-0.333* 


-0.162 


0.642*** 


-0.648** 


1.117*** 


1,877 


0.011 


3 

O 


26 


Paraprofessionals per teacher 


0.003 


0.036* 


0.091*** 


-0.043* 


0.123*** 


2,170 


0.036 


(0 

0> 


27 


Share of teaching staff providing mentoring support 


-0.006 


-0.015*** 


0.011*** 


0.024*** 


0.007*** 


2,190 


0.037 


QC 




28 


Share of teaching staff providing instructional support 


-0.004* 


-0.004 


0.008*** 


-0.003 


0.001** 


2,190 


0.017 






29 


Administrators + clerical office staff per principal 


7.001*** 


3.504*** 


-5.362*** 


10.678*** 


9.714*** 


1,971 


0.049 




Instructional 

emphasis 


30 


Share of total teachers who teach core subjects 


-0.024 


0.055*** 


-0.005 


0.076*** 


0.586*** 


2,190 


0.017 




31 


Share of total teachers who teach elective subjects 


0.028*** 


-0.02** 


-0.101*** 


-0.008 


0.234*** 


2,190 


0.216 




32 


Elective teachers per core teacher 


0.052 


-0.058 


-0.179*** 


-0.051 


0.418*** 


2,165 


0.056 




Teacher status 


33 


Share of total teachers designated as probationary or temporary 


-0.076*** 


-0.012 


0.055*** 


0.084** 


0.232*** 


2,190 


0.026 




& 


34 


Share of total teachers with tenure 


-0.08** 


-0.209*** 


-0.21*** 


-0.119*** 


0.768*** 


2,190 


0.127 




characteristics 


35 


Share of total teachers with full credentials 


-0.014 


-0.108*** 


-0.044*** 


-0.125*** 


0.962*** 


2,190 


0.150 


* significant at 10%; 


** significant at 5%; *** significant at 1% 
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Staff size relative to enrollment: Do BTO schools have more personnel resources than other 
schools? 

Across elementary, middle and high schools, the results from the regression model provide no 
statistical evidence indicating that BTO schools and low-performing schools have different 
quantities of certified and classified personnel when compared to other public schools. 

Staff experience and education levels: Are teachers and administrators in BTO schools more 
experienced and more highly educated? 

When controlling for student characteristics, the education and experience levels of teachers in 
BTO schools are no different than their counterparts in other public schools. However, 
differences in the level of experience and education displayed by administrators in elementary 
BTO schools still holds. As shown in Exhibit 4.1.6, administrators in elementary BTO schools 
show a statistically significant difference of 2.6 more years of experience as educators (Variable 
#13, Column A) and 4.5 more years of experience in the district (Variable #14, Column A) 
compared to administrators in other public elementary schools. 

In contrast, both teachers and administrators in low-performing schools have levels of experience 
in education that are statistically significantly lower than the state average (Variables #10 and 13, 
Column B). Teachers in low-performing elementary schools are approximately one year less 
experienced in education and are one year newer to the district relative to teachers in other public 
schools. 

As shown in Exhibit 4.1.7, across middle and high schools a similar pattern emerges: LP school 
teachers have lower educational attainment (Variable #9, Column B), are approximately two 
years less experienced as educators (Variable #10, Column B), and are one year newer to the 
district (Variable #1 1, Column B) relative to teachers in other public schools. Administrators in 
LP schools have two fewer years of experience in education as well (Variable #13, Column B). 

Staff distribution: Do BTO schools distribute their staff differently aeross the major assignments 
(i.e., teaching, administrative and pupil support positions) relative to other schools? 

As shown in Exhibit 4.1.6, in elementary schools, there are no significant differences in the 
proportion of total staff who are teachers between BTO, LP and other public schools (Variable 
#21, Column A and B). In the case of administrators, BTO schools have a higher proportion of 
staff in administrative positions compared to the rest of schools (Variable #22, Column D). In 
addition, LP schools have a higher proportion of staff in pupil support assignments (Variable 
#23, Column B) when compared to other public schools. 

As shown in Exhibit 4.1.7, in the case of middle and high schools, the only significant difference 
that is observed is the higher pereentage of staff in BTO schools providing pupil support 
services. LP middle and high schools resemble other public schools 
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Mutual support capacity: Do BTO schools have more capacity to support their teachers? 

Yes. Regarding elementary schools, the most salient difference is in FTE teachers per 
administrator. BTO schools have 4.4 fewer teachers per administrator ((Variable #24, Column 
A) than other public schools, or in other words, more administrators per teacher. In addition, 
middle and high school BTO schools have more people support per teachers (Variable #24, 
Column A) relative to other public schools. 

Principal support capacity: Do BTO schools have more capacity to support their principals? 

Yes. Regarding the capacity of schools to support principals, at the middle and high school level, 
the results indicate that both BTO schools and LP schools have more administrators (e.g., vice 
principals) and clerical office staff per principal compared to other public schools (Variable #29, 
Columns A and B) . 

Teachers’ status: Do BTO schools have a lower percentage of teachers with tenure? 

Yes. At the elementary school level, BTO schools have a significantly lower percentage of 
teachers with tenure (Variable #32, Column A) when compared with LP and other public 
schools. LP schools do not differ from other elementary schools on this measure. In the case of 
middle and high schools, BTO and LP schools have a lower percentage of tenured teachers 
((Variable #34, Columns A and B). 

Teacher credentials: Do BTO schools have a higher percentage of teachers with full credentials? 

BTO elementary and middle or high schools are not statistically different from other public 
schools. In contrast, LP elementary, middle and high schools have significantly lower percent of 
teachers holding full credentials (Variable #33, Column B in Ex. 4.1.6 and Variable #35, Column 
B in Ex. 4.1.7, respectively). 

Class sizes and case loads: Do BTO schools have smaller class sizes? 

BTO schools do not have smaller classes in grades 1 to 5. However, BTO schools have two 
fewer students in their kindergarten classrooms (Variable #15, Column A) compared to other 
public schools. Among middle and high schools, BTO schools have higher case loads in core 
subject classes (Variable #15, Column A) compared with other public and LP schools. 

Summary 

The regression results provide no statistical evidence that BTO and LP schools have different 
quantities of certified and classified personnel when compared to other public schools. Also, 

BTO schools do not have smaller classes in grades 1 to 5 compared to other public schools (BTO 
do have two fewer students in their kindergarten classrooms; see Variable #15, Column A). The 
education and experience levels of teachers in BTO schools do not differ compared to other 
public schools, but the years of experience of elementary BTO administrators is higher 
(Variables #13 & 14, Column A) than their colleagues in other public schools. In contrast, both 
the level of experience of teachers and the administrators’ educational attainment in elementary 
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LP schools are lower (Variables #10, 11 & 12, Column B) than the state average. Further, 
teacher education and experience is lower (Variables #9, 10 & 11, Column B) in low performing 
middle and high schools than in other public schools. 

In elementary schools, BTO schools have a higher proportion of staff in administrative positions 
(Variable #22, Column A) compared to the rest of schools. In addition, LP schools have a higher 
proportion of staff in pupil support assignments (Variable #23, Column B) when compared to 
other public schools. 

Also, BTO schools have a significantly lower percentage of teachers with tenure (Ex. 4.1.6, 
Variable #32, Column A; Ex. 4.1.7, Variable #34, Column A) when compared with other public 
schools. Last, LP elementary, middle and high schools have significantly lower percentages of 
teachers holding full credentials (Ex. 4.1.6, Variable #33, Column B; Ex. 4.1.7, Variable #35, 
Column B). 

Section II: Results of Costing Out BTO and LP Schools 

This section presents the results of costing out BTO and LP schools. As mentioned previously, 
our intent is not to estimate an “adequate” total education expenditure level for all schools in 
California based on these schools, but to analyze how BTO and LP schools contrast in their level 
of spending. These estimates are also compared with statewide averages of total per pupil 
spending. 

As detailed in the methodological chapter (Chapter 3, Section II-B), at the center of our costing- 
out exercise lies a teacher and an administrator wage equation. The results of the estimations of 
these two equations are presented below (Exhibit 4.2.1). 



Exhibit 4.2.1. Weighted Least Squares Regression Results for Teachers and Administrators 





Teacher Salary Equation 


Administrator Salary 
Equation 


Average education level**** 


5,027 ** 


6,250 ** 


Average experience level 


3,742 ** 


613 


Average experience squared 


-115 ** 


-7 


Wage Index *'** 


27,512 ** 


61,321 ** 


Constant 


3,283 


12,581 


Number of observations 


1,027 


979 


R-squared 


0.11 


0.13 



* significant at 5%; ** significant at 1% 

(a) Average educationai ievei is centered at 16 years. This means that the expected saiary for a teacher in LAUSD with 16 years of 
education and 1 0 years of experience wiii be (5,027 * 0) + (3,742 * 1 0) - (1 1 5 * (1 0)''2) + (27,51 2* 1) + 3,283 

(b) The Wage index is centered on LAUSD; it wiii take the vaiue of 1 when the iocation is LAUSD. 



The weighted least squares result for teacher wages shows that all regressors are statistically 
significant and have the expected sign. The average teacher salary increases with higher levels of 
teacher education and experience, and the marginal effect of experience decreases when teacher 
experience increases (reflected in the negative sign of experience squared). In the case of the 
salary equation for administrators, only the education level is statistically significant. 
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Exhibit 4.2.2 presents the results of the costing out methodology. It shows that, on average, when 
using this standardized approach to costing out education, beating-the-odds schools appear to 
spend slightly less than low-performing schools per student ($7,799 vs. $8,021). Also, in the case 
of BTO and LP schools, it shows total per pupil spending as positively correlated with poverty. 

In other words, high poverty schools tend to show higher per pupil spending. In addition, high 
poverty BTO schools show higher per pupil spending than high poverty LP schools, with the 
reverse showing when lower poverty schools are compared. 



Exhibit 4.2.2. Totai Weighted Per Pupii Spending; BTO and LP Schoois (2004-05) 





Total per 
Pupil 
Spending 


Total 

Personnel 
per Pupil 
Spending 


Central 
Office per 
Pupil 
Spending 


Plant per 
Pupil 
Spending 


Other 
Services 
per Pupil 
Spending 


Non- 

Personnel 
per Pupil 
Spending 


Overall 






Beating the odds (n=103) 


7,799 


5,554 


371 


751 


699 


425 


Low performing (n=1 13) 


8,021 


5,603 


366 


782 


802 


468 


Below Median Poverty 
(0%-71%) 






Beating the odds (n=72) 


7,648 


5,429 


377 


739 


688 


414 


Low performing (n=37) 


7,743 


5,391 


376 


766 


779 


431 


Above Median Poverty 
(< 71% -100%) 






Beating the odds (n=31) 


8,458 


6,097 


345 


801 


747 


469 


Low performing (n=76) 


8,192 


5,728 


375 


804 


880 


499 



Source: California Department of Education, SACS and CBEDS data 2004-05. 



Exhibit 4.2.3 shows total per pupil spending for the state of California. When compared with 
beating-the-odds and low-performing schools, total expenditures do not seem to differ 
noticeably. The statewide per pupil expenditure is $7,523; BTO school’s level of spending is 
slightly higher than the state average (about $260 per pupil), and LP schools spend an average of 
about $500 per student more than the average school in the state. 



Exhibit 4.2.3. Statewide Average Expenditures, 2004-05 



Function 


Average per ADA 
Expenditures 
All Districts 


Instruction 


4,687 


Instruction reiated services 


930 


Pupil services 


523 


General administration 


395 


Plant services 


729 


Other 


69 


Total per pupil spending 


7,523 



Source: Ed-Data, Education Data Partnership. Statewide Financial Report, 2004-05. 
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Summary 

We have seen in this analysis that standardized spending in BTO and LP schools differs slightly: 
BTO schools appear to spend $222 per student less annually than LP schools. Also, both sets of 
schools are shown to spend somewhat more than the average spending level in the state. 

Section III: Resource Allocation as an Optimization Problem: 
Results 

In this section, we attempt to relate resource allocation practices and student demographic 
characteristics to differences in student academic attainment. We learned from the previous 
sections that levels of resources and spending in BTO schools do not differ substantially from 
other public schools, and that teachers in BTO schools have higher levels of experience and 
education than teachers in LP schools. 

In this section, we estimate the model presented in Chapter 3, Section II-C, and then present 
analyses of the extent to which observable differences in resource allocation practices as well as 
student characteristics among BTO, LP and the rest of public schools are able to explain their 
differences in average academic achievement. We conclude this section with an attempt to 
analyze how efficient BTO and LP schools are in their use of resources. In particular, we focus 
on the resources spent on teacher salaries. 



The Academic Production Modei 

Using statewide data we estimate the following model (Equation (3) from Chapter 3): 



— —a /S tea 



Max L = A edu exp'^ ( Y pov els enr 

enr 



The 2005 California Standards Test (CST) results for English language arts (ELA) and 
mathematics are used as a measure of academic outcome ( T ), as well as the average residual 

and success rate obtained when BTO and LP schools were selected. CBEDS and other extant 
statewide data are used to measure enrollment (enr), and the number of teachers (PTEs) at each 
school (tea) and their educational attainment and teaching experience. With this information, we 

estimate the average years of teacher experience ( exp ) and education ( edu ) for all teachers at 
each school, variables on the percentage of students eligible for free or reduced price lunch (a 
proxy for poverty) and the percentage of English learners are also included using other extant 
data sources. 

Exhibit 4.3.1 shows the OLS results when the CST results for ELA and mathematics are used as 
a measure of academic outcome.^° All regressors are statistically significant at 1 percent, and 
have the expected sign. Average school-level academic outcome increases with higher teacher 
education and higher teaching experience, as well as with increases in the ratio of teachers per 



In the case of mathematics, the regression has 6,467 observations, in the case of ELA, there are 7,395 
observations. These differences are explained due to the lack of a consistent CST mathematic test for high schools. 
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students. In addition, with increases in the poverty level — as measured by the percentage of 
students eligible for free or reduced price lunch and/or who are English learners, the average 
school-level performance decreases. The R-squared values are very high for cross-sectional 
regressions, at 0.6 for CST mathematics and 0.7 for CST ELA. 



Exhibit 4.3.1. OLS Regression of Average Schooi-Levei Academic Achievement 





Standardized CST 
Math 


Standardized CST 
ELA 


Ln (Average years of teaching experience) 


0.260 

(0.022)*** 


0.285 

(0.018)*** 


Ln (Average years of teacher education) 


1.287 

(0.305)*** 


1.239 

(0.232)*** 


Ln (teachers per student) 


0.131 

(0.047)*** 


0.346 

(0.036)*** 


Ln (Percentage of students eiigibie for free or 


-0.684 


-0.655 


reduced price iunch) 


(0.010)*** 


(0.008)*** 




-0.032 


-0.1 15 


Ln (Percentage of English learners) 


(0.008)*** 


(0.007)*** 


Constant 


-4.544 

(0.852)*** 


-3.957 

(0.649)*** 


Observations 


6,467 


7,395 


R-squared 


0.6213 


0.7059 



* Significant at 10%, ** significant at 5%, *** significant at 1% (p-values in parenthesis). 



Exhibit 4.3.2 shows the OLS results when multiple years are used to measure academic outcome. 
This is the case when the average residual and success rate are used as a measured of academic 
performance. These regressions include additional regressors as well. Eor example, the number 
of administrators and classified personnel per student is included. The average years of 
experience and education for administrators as well as the percentage of teachers in probationary 
and tenure status are incorporated in the regressions. The results show that in both regressions 
almost all regressors are statistically significant and have the expected sign. Eor example, the 
results for the average residual show that higher levels of personnel (e.g., teachers and 
administrators) are related with a higher average residual. The same is true for teacher and 
administrator education and experience level; higher the level of those variables higher the 
average residual. 

When the success rate is used as a measure for academic outcome, the results are similar. One 
exception is the sign of the variable that measures the percentage of teachers with tenure status. 
Higher the percentage of teachers with tenure, lower the success rate measure. When the average 
residual is used as a measure of performance the reverse is observed. Another exception is the 
teacher pupil ratio variable. When the success rate is used as outcome measure, this variable is 
not statistically significant (the opposite is observed when the average residual is used as 
dependent variable). 



These measures were introduced in Section I of Chapter III. The average residual was calculated across years and 
subjects (i.e., ELA and math) from the regression results obtained when BTO and LP schools were selected. The 
success rate was calculated by measuring the number of requirements a school met every year to be considered 
BTO. For example, a school that met all the requirements during the four-year period has a success rate of 1 and is 
considered a BTO school. Schools that missed all the requirements have a success rate of zero and are considered 
LP schools. 
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In addition, it is important to observe that the R-squared values are much lower than when only a 
one year measure of academic performance is used (Exhibit 4.3.1). The R-squared is 0.395 for 
the average residual regression, and 0.138 for the success rate regression. This means that 
resources and student characteristics do not do a good job of explaining the performance of 
schools that are consistently beating the odds during a period of time longer than one year. It is 
possible that the factors that make schools successful are unobservable characteristics, or are not 
captured by current statewide databases. For example, indicators associated with school 
leadership, teachers’ planning time, and teacher and principal turnover are not uniformly 
available to conduct a comprehensive analysis for all schools in the state. 



Exhibit 4.3.2. Expanded OLS Regression of Average Schooi-Levei Academic Achievement 





Average Residual 


Success Rate 


Ln (Teachers per student) 


0.873 

(0.403)** 


-0.016 

(0.032) 


Ln (Administrators per student) 


1.484 

(0.118)*** 


0.067 

(0.009)*** 


Ln (Number of ciassified personnel) 


0.857 

(0.118)*** 


0.029 

(0.009)*** 


Ln (Average years of experience - teachers) 


1.665 

(0.198)*** 


0.098 

(0.016)*** 


Ln (Average years of experience - administrators) 


-0.030 

(0.080) 


0.006 

(0.006) 


Ln (Average years of education - teachers) 


6.070 

(2.338)*** 


0.585 

(0.186)*** 


Ln (Average years of education - administrators) 


-2.723 

(0.977)*** 


0.074 

(0.078) 


Ln (Percentage of teachers in probationary status) 


0.784 

(0.064)*** 


0.010 

(0.005)** 


Ln (Percentage of teachers with tenure) 


0.329 

(0.088)*** 


-0.039 

(0.007)*** 


Ln (Percentage of students eligible for free or 


-0.005 


-0.009 


reduced price lunch) 


(0.064) 


(0.005)* 


Ln (Percentage of English learners) 


-0.415 

(0.053)*** 


-0.025 

(0.004)*** 


Ln (School size) 


4.525 

(0.092)*** 


0.153 

(0.007)*** 


Constant 


-26.139 

(6.432)*** 


-2.54 

(0.513)*** 


Observations 


5,765 


5,765 


R-squared 


0.395 


0.138 



* Significant at 10%, ** significant at 5%, *** significant at 1% (p-values in parentheses). 



Academic Achievement and Resource Aiiocation 

In Section I of this chapter we analyzed various resource measures of BTO, low-performing, and 
other public schools. The results did not show substantial differences between BTO, LP, and the 
rest of public schools. The schools have similar levels of personnel (i.e., FTEs of teachers, 
administrators, and pupil support), and BTO schools’ teachers have similar experience and 
education levels to teachers in other public schools. In addition, differences in teacher pupil 
ratios are very small. However, it was shown that FP schools have teachers with less experience 
in education, and with lower levels of educational attainment compared to other public schools. 
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Furthermore, in the previous section we showed that there seems to be a relationship between 
some measures of resources and student characteristics and academic outcomes (Exhibit 4.3.1 
and Exhibit 4.3.2). In this section we study the extent to which resource differences may explain 
differences in academic achievement among. Using the OLS regression coefficients displayed in 
Exhibit 4.3.1, we estimate how certain levels of resources and student characteristics explain 
differences in the academic performance level of BTO, LP, and other public schools. These 
exercises use only one year of academic outcome measure (standardized CST ELA and math 
results for 2005). This is estimated in the following way: 

Effect on Achievement = {\n{Avg.lnputBjg)- \n(Avg. Input 

where Avg. Input is measured by the average years of teacher education, teaching experience, and 
class size. 

Exhibit 4.3.3 shows the average years of teacher education, teaching experience, and class size at 
BTO and other public schools (i.e., not including LP schools). These differences in average 
inputs can be used to generate an effect on achievement (or achievement gap) between these 
schools. This is contrasted with the actual gap in CST mathematics scores that exists between 
these two groups of schools. 



Exhibit 4.3.3. Expiained Difference in CST Mathematics Scores between BTO and Other Pubiic 
Schools (2004-05) 







BTO 

Schools 


Other Public 
Schools 

(Excluding LP 
Schools) 


Effect on Student 
Achievement 
CST Mathematics 

(In Standard 
Deviations) 






1 


2 


3 


A 


Average Years of Teacher Education 


17.2 


17.3 


-0.003 


B 


Average Years of Teaching Experience 


12.6 


12.9 


-0.007 


C 


Average Class Size - Grades 4-5 


29.3 


30.4 


0.005 


D 


Total Effect on Achievement due to Resources (A3 + B3 + C3) 






-0.005 


E 


Poverty Level 


0.6 


0.5 


-0.065 


F 


Percent English Learners 


0.3 


0.3 


-0.004 


G 


Difference due to Demographics (E3 + F3) 






-0.069 


H 


Total Predicted Difference (D3 + G3) 






-0.073 


Observed Difference in 
Student Achievement 


1 


Average Standardized CST Math Score 


1.05 


0.04 


1.01 


J 


Achievement Difference Explained by Resources (D3/I3) 






0% 


K 


Achievement Difference Explained by Demographics (G3/I3) 






0% 



These results suggests that our ability to explain the difference in average mathematics 
achievement between BTO and the rest of public schools in California through differences in 
teaching experience, educational level, and class size is extremely limited (cell D3). Eor CST 
mathematics, the total estimated effect is 0.005 standard deviations m favor of other public 
schools. That is, the differences in resources between these two sets of schools actually predict a 
higher average test score for other public schools instead of BTO schools. When we also take 
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into consideration differences in student characteristics, the total estimated effect increases to 
0.073 standard deviations (cell H3). This prediction is in sharp contrast with reality, given that 
BTO institutions actually outperform other public schools by more than a full standard deviation 
(cell 13). Given this disparity, the percentage of the academic difference or gap that exists 
between BTO and other public schools (i.e., 1.05 - 0.04 = 1.01) that is explained by differences 
in resources and by differences in the demographic characteristic of these two sets of schools is 
zero (cells J3 and K3). 

Similar results were obtained when we used the CST English language arts as a measure of 
academic achievement. Differences in resources are able to explain a very small portion of the 
academic performance difference between BTO and other public schools. In addition, 
differences in student characteristics explain none of the higher academic level; in fact, the 
student characteristics of these schools predict a lower academic achievement level in these 
schools relative to the other public schools. 

Next, we analyze how resources and student characteristics explain the low academic 
performance level of LP schools. 
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Exhibit 4.3.4 presents the same exercise for low-performing (LP) schools. Differences in 
resources account for a small portion, 5.1 percent (cell J3) of the 1.16 standard deviation (cell 13) 
difference in math achievement between students in LP and all other public schools. Student 
characteristics, however, explain a much larger proportion of the achievement gap in this case, at 
25.8 percent (cell K3). When we combine what we are able to explain through differences in 
resources and student characteristics, our ability to explain the achievement gap between LP and 
other public schools increases to 30.9 (5.1 -1-25.8) percent. 



Exhibit 4.3.4. Expiained Difference in CST Math Scores between LP and Other Pubiic Schools 
(2004-05) 







LP 

Schools 


Other Public 
Schools 

(Excluding BTO 
Schools) 


Effect on Student 
Achievement 
CST Math 

(In Standard 
Deviations) 






1 


2 


3 


A 


Average Years of Teacher Education 


17.09 


17.27 


-0.013 


B 


Average Years of Teaching Experience 


10.71 


12.92 


-0.049 


C 


Average Ciass Size - Grades 4-5 


29.76 


30.43 


0.003 


D 


Difference due to Resources (A3 + B3 + C3) 






-0.059 


E 


Poverty 


0.77 


0.51 


-0.285 


F 


Percent Engiish Learners 


0.39 


0.25 


-0.014 


G 


Difference due to Demographics (E3 + F3) 






-0.299 


H 


Totai Predicted Difference (D3 + G3) 






-0.358 


Observed Difference 
in Student 
Achievement 


1 


Average Standardized CST Math Score 


-1.12 


0.04 


-1.16 


J 


Achievement Difference Explained by Resources (D3/I3) 






5.1% 


K 


Achievement Difference Explained by Demographics (G3/I3) 






25.8% 



When ELA results are used as a measure of academic outcome, similar results are obtained. 
About 6 percent of the academic difference between the two school types is explained by 
resources and about 30 percent is explained by the variations in student demographics. Thus, for 
these schools, at least some of the observed outcome disparity appears related to variables such 
as general teacher characteristics such as education and experience. Lor BTO schools, however, 
these variables appear virtually unrelated to these schools unusually high academic achievement 
levels. Lor these schools, our ability to predict their unusually high performance through this 
type of approach is very limited. 

In the next chapter we present results gathered through telephone interviews for a sample of 
BTO and LP schools. These provide added insight in regard to factors that are not easily and 
uniformly measured. Before turning our attention to the results of our telephone interviews, 
however, the next section provides an analysis of efficiency of BTO and LP schools. 
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Addressing the Efficiency Question 

A common criticism of exercises that try to link resources to student outcomes is that they 
generally do not address efficiency. As Hanushek (2005) states, “The existing analyses never 
consider the minimum cost, or efficient level of spending, needed to achieve the desired 
outcome.” Above, we have provided estimates of average per pupil expenditure for BTO and LP 
schools in California. But what do they mean? Do they mean that any school with certain student 
demographics will reach the performance of BTO schools if their level of spending is similar to 
these schools? The evidence provided by LP schools suggests otherwise, that education spending 
is not sufficient or provides no guarantee of academic success. This finding may be at least 
partially explained by inefficient use of resources at these schools. In other words, is there room 
for improving student achievement without increasing expenditure? Or is there room to reduce 
expenditure maintaining student achievement intact? How much room for improvement do BTO 
and LP schools appear to have? 

In this section we focus on this last question, and in particular we analyze whether BTO and LP 
schools appear to be efficient in regard to the resources spent on teacher salaries. In order to 
answer this question we have to consider both the implications of the estimated production 
function (Exhibit 4.3.1), as we did in the previous section, and the relative costs of the different 
inputs. Taking into account the relative marginal productivity of resources, as well as their 
relative costs, provides a basis for analyzing the efficiency of these schools. 

The research question we try to answer here is whether there is any evidence that BTO schools 
use their teacher resources in a more efficient way than LP schools. In particular, we concentrate 
on the optimal ratio between teacher experience and education. It is worthy to note that this 
question is relevant to the extent that schools/districts have freedom to choose a preferred mix of 
teacher experience and teacher education. 

The first order conditions of exp and edu of the model detailed in Chapter 3, Section III-C, are 
the following: 



ain(L) P 

= ^ — X {02 tear) = 0 

oexp exp 


(6) 


ai^)^_« /l(^jfear)=0 
dedu edu 


(V) 
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Therefore, the optimal ratio of teacher experience to teacher education is equal to: 



( 8 ) 

edu a 62 

It is important to state an important limitation of this model. This optimum level is a partial 
equilibrium and is not able to take general equilibrium effects into account. In other words, the 
model requires that each school represent only a small portion of the total pool of schools. This 
means that in our analysis only BTO and LP schools are analyzed. To understand this limitation, 
consider an analysis of the current and optimum usage of resources for all public schools in the 
state. If we learned that all of these schools were operating at a non-optimal point, how could all 
public schools in California hire more or less experienced teachers given that we know that the 
amount of teaching experience is relatively fixed within the state, at least in the short to medium 
term? 

Now, in order to estimate how close BTO and LP schools are operating in regard to this optimum 
ratio of teacher education and teaching experience, we must estimate the wage equation 
presented in Chapter 3 (Equation 5). 

As detailed in the methodological chapter, we used data from the Standardized Account Code 
Structure (SACS) and the Charter School Alternative Form Database of 2004-05 to estimate the 
wage equation. One challenge of this analysis is that it combines district- and school-level 
information. In order to estimate Equation (5) it is necessary to first divide each district's total 
expenditure on teacher salaries by the number of schools (including those charter schools using 
SACS) operating in each district. This gives us an estimate of teacher salary expenditure at the 
school level. The same procedure is used to get an estimated average of total number of years of 
teacher education and teaching experience for these schools.^^ 

The wage equation that we estimate in this section is presented below. Note that we have 
centered the variable of years of education around 16 in order to get a constant that is easier to 
interpret. 



W = 6 q +0^ (edu-l 6)+02 exp -I- rwi 



Exhibit 4.3.5 shows the results of estimating the wage equation by weighted least squares. The 
weight is given by the number of schools operating in each district.^^ We have also included a 
relative wage index (rwi) in order to take geographic and urban-rural wage differences into 
account (the same that was used to costing out BTO and LP schools). 



Note that from CBEDS we can obtain actual (not average) figures of total number of years of teacher education 
and teaching experience for these schools. But given that the dependent variable — average expenditure in teacher 
salaries — does not vary within districts, this within-district variation of teacher education and teaching experience of 
schools reporting in SACS is not relevant for this wage regression. 

In the case of charter schools using the Alternative Form their weight is equal to one. 
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The weighted least squares results indicate that each additional year of average teacher education 
increases the average wage by about $3,574. In contrast, one additional year of average teaching 
experience increases the average wage by about $771. 



Exhibit 4.3.5. Weighted Least Squares Regression of Average Schooi-Levei Teacher Wage 





School-Level Average 
Teacher Salary 


Average years of teacher education*®* 


3,574 

(0.007)*** 


Average years of teaching experience 


771 

(0.000)*** 


Relative wage index 


30,548 

(0.000)*** 


Constant 


21,489 

(0.000)*** 


Observations 


1,229 


Population size 


9,224 


R-squared 


0.10 



(a) Average educational level is centered at 1 6 years. 



Combining the empirical estimates of the production function and wage equation, and the 
theoretical optimum ratio of teacher education versus teaching experience, we can now address 
the efficiency issue. 

Exhibit 4.3.6 presents the isoquant of BTO schools for CST math. This line shows how the 
average 2004-05 academic achievement of BTO schools in math could be maintained using 
different combinations of teaching experience and education. This isoquant is drawn using the 
average class size observed in grades 4 and 5, as well as the average level of poverty and 
percentage of English learners served by these schools. In addition, BTO schools have an 
average teacher experience of 12.59 years and an average education of 17.23 years. This means 
that the ratio at which they are using these two resources is equal to 0.73 (12.59/17.23) units of 
teaching experience per unit of teacher education. 

Equation (8) states that the optimum ratio is equal to: 

e^ ^ ^ 3^ 

edu ae^ 1.287 771 

A ratio smaller than 1 means that even though each additional year of teacher education has a 
higher cost (i.e., has a larger effect on teacher wages) than an additional year of teaching 
experience, its greater impact on academic achievement more than offsets this differential cost. 

In other words, it is optimal to invest slightly more in education than experience. 

Exhibit 4.3.6 presents the actual ratio for BTO schools, the optimum ratio, and where LP schools 
are in this mix of teaching experience and teacher education. As observed, BTO schools use a 



Overall averages for BTO and LP schools (i.e., for elementary, middle and high schools combined) are presented 
in Appendix 4. 
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lower ratio of years of teaching experience to years of teacher education than what would be 
optimum at the current relative prices. As shown, relative prices of these inputs are tangent to the 
isoquant at the point represented by 15.13 average years of teaching experience and 16.09 
average years of teacher education. This mix of teacher education and experience is the one that 
minimizes the costs of these two inputs while maintaining exactly the same academic 
achievement level reached by the current usage of inputs (i.e., 12.59 and 17.23). 

Exhibit 4.3.6. Isoquant of BTO Schoois and Optimum Leveis of Teacher Education and Teacher 
Experience. 




Average Teaching Experience 



Isoquant Relative Prices 

• Optimum Allocation • BTO Allocation 



We have observed that BTO and LP schools are not operating at the optimum level that is 
predicted by our model, however, BTO schools are much closer to the optimum when compared 
with LP schools. We can now analyze the impact on monetary resources if BTO and LP schools 
were to move to the optimum level. 

Exhibit 4.3.7 shows the results of this exercise for BTO schools. At a marginal cost of $3,574 for 
each year of teacher education and $771 for each year of teaching experience (wage equation 
results), the current mix of these two resources has a monetary cost of $71,325 per FTE. If BTO 
schools used the optimum ratio of teaching experience to teacher education, they would face a 
monetary cost of $69,216 per FTE. The potential gain in efficiency is $2,108 per FTE. 
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Exhibit 4.3.7. Potentiai Efficiency Gains for BTO Schoois 





Average Years of 
Teacher 
Education 


Average Years of 
Teaching 
Experience 


Monetary Cost 
of Inputs 


BTO schools 


17.23 


12.59 


$71,287 


Optimum 


16.09 


15.13 


$69,171 


Potential efficiency gain per FTE 






$2,116 



Even though this amount may seem relatively low, it becomes more significant when we take 
into account the average number of teachers employed by these schools. The average number of 
FTE in teaching assignments in BTO schools is 54, which means that, on average, a BTO school 
can save $1 14,265 a year by hiring a teacher work force with slightly more experience and a 
slightly lower education level, while maintaining the same level of academic achievement. 

Exhibit 4.3.8 shows the results of this same exercise for LP schools. It shows that the potential 
gains from moving to an optimum resource allocation are much larger for LP schools. In other 
words, these findings imply that LP schools are substantially less efficient than BTO schools. 
Their current mix of teacher education and teaching experience has a monetary cost of $69,337. 
If these schools were able to employ more experienced but less educated teachers, these findings 
suggest that they would be able to maintain their current average academic achievement while 
saving $10,264 per FTE. Since the average number of teachers per LP school is around 50, this 
would translate into an annual efficiency gain of $513,201 per school. 



Exhibit 4.3.8. Potential Efficiency Gains for LP Schools 





Average Years of 
Teacher 
Education 


Average Years of 
Teaching 
Experience 


Monetary Cost 
of Inputs 


LP schools 


17.09 


10.71 


$69,337 


Optimum 


13.75 


12.88 


$59,073 


Potential efficiency gain per FTE 






$10,264 



Summary 

We have found that the actual levels of teacher education and teacher experience in BTO schools 
are closer to the optimum predicted by the model, or in other words, BTO schools appear to be 
operating at a more efficient level than LP schools in regard to their teacher characteristics. In 
addition, these findings suggest that if BTO and LP schools were to move to the optimum level 
predicted from the model, they could gain in efficiency that would translate in monetary savings. 
Also, LP schools show a much higher potential gain in efficiency if they were to move closer to 
the optimum ratio of teacher experience to teacher education. It is important to keep in mind that 
in the case of LP schools this would result in a gain in efficiency, but not in an improvement of 
their academic performance level. 
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Introduction 

This chapter focuses on the results of the telephone interviews conducted with BTO schools and 
comparison schools. In particular, it details what has enabled BTO schools to consistently 
produce substantially higher educational outcomes than other schools like them. The analyses 
presented in Chapter 4 suggest that the answer is not more resourees. In the aggregate, the 
resources available to these schools appear similar to those of their lower-performing 
counterparts. Perhaps their mix of resources and how they use them is important to their success. 

Knowing that BTO schools do not appear to have more resources than other schools seems an 
important finding. Still, it begs the question of what does make a difference in BTO schools. 
Statewide data do not provide sufficient depth to provide much insight into one of the primary 
research questions for this study: What resource allocation patterns, instructional practices, 
programmatic leadership, or professional development opportunities are observed or reported by 
schools as being related to their academic success? 

In an attempt to dig deeper into the faetors that seem to have made a difference for BTO schools, 
we conducted phone interviews with 18 BTO principals. As points of comparison, we also asked 
similar questions of principals from five “comparison” schools, or in other words, schools that 
were doing much worse than would be expected given the student populations they serve. As we 
heard certain faetors stressed by BTO principals, it was interesting to note the extent to which we 
heard low-performing principals espouse similar strategies or factors for success. 

Furthermore, it seems clear that at least among these schools, there is not a single key to 
academie success. It is not even clear that there is a single combination of factors, or a single 
recipe. In many ways, the combination of factors said to be major eontributors to the success of 
the BTO schools seemed unique to each school. On the other hand, some factors were mentioned 
frequently enough to emerge as themes. Some of the factors cited by the BTO schools were also 
mentioned by the comparison schools interviewed. A comparison school’s principal citing the 
same factor that a BTO principal cites does not mean the factor is not valid. The faet that both 
high- and low-performing principals acknowledge the importance of a certain element (e.g., 
control over teacher hiring and retention) may simply underline the importance of this 
component and the fact that perhaps only some principals have the ability to actually do it. 

In short, what we have found is that the answer to success across the BTO sehools included in 
this study is complex. It is not simply more resources or the application of a certain recipe in 
regard to resource allocation practices. Although we did not find the answer to school success 
through these interviews, we were able to discern a eonstellation of faetors cited as leading to 
success among the BTO schools. What we found in this analysis is not new. Our findings are 
similar in many ways to those of other research into the differences between relatively successful 
and unsuccessful schools serving eomparable student populations. 
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Findings from the School Interviews 

As detailed in Chapter 2, research on effective practices has found that successful schools share a 
common set of characteristics. This chapter builds on existing research that has examined the 
effects of some of these components on student learning (Purkey and Smith, 1983; Levine and 
Lezotte, 1990; O’Day, Goertz, and Floden, 1995; Newmann and Wehlage, 1995; Mohrman and 
Lawler, 1996; Williams, Kirst, & Haertel, 2005; Oberman et ah, 2005; Parrish, Merickel, Perez, 
Linquanti, Socias, Spain et ah, 2006). We end this section by exploring the challenges that 
schools faced in their efforts to improve their students’ education, as well as their perceptions in 
regard to the need for additional funding to achieve California’s academic goals. 

A theoretical framework of the strategies for school success from the school interviews is shown 
below. The framework also depicts how these components relate to each other. Because we 
wanted to provide these schools with as much latitude as possible to discuss factors contributing 
to their unusually high test scores, questions were asked in an open-ended manner. Given this, 
we gathered the information that the principals chose to tell us, instead of a complete list of all 
the components inherent to their success. Just because a principal failed to mention an element, 
we cannot conclude that this element was not important at the school. At the same time, the 
open-ended structure of this approach allowed them to focus on what they felt was most 
important. After presenting the theoretical framework, we discuss each component of the 
framework through concrete examples of how schools implement them, and discuss factors 
influencing their implementation. 



Theoretical Framework 

Cohen and Ball (1999) described the complexities of school capacity, mentioning that capacity is 
“a function of the interaction among [teachers, students, and educational materials]” and not a 
function of merely one factor such as teachers’ skill or curriculum. In order to understand 
instructional capacity, it is important to view the organization and environment in which these 
factors exist. 

Exhibit 5.1 displays a theoretical framework that helps us understand the relationship between 
ten different factors that contribute to educational success, as well as the role school and district 
leaders and external consultants have in influencing school performance. This framework was 
developed from the strategies for success that were described by the principals of 18 BTO 
schools, using interviews from principals at 5 low-performing schools as points of comparison. 
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Exhibit 5.1 Theoretical Framework 
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As shown in the theoretical framework diagram, the major factors identified by the respondents 
for this study were: 

1) Existence of high-quality teachers and staff; 

2) Implementation of a standards-based curriculum; and/or 

3) Coherent instruction. 

The existence of high-quality teachers and staff in the schools is the result of many confounding 
factors. For example, the support that teachers receive and the control over who is teaching at the 
schools seem to be critical features that would lead to high-quality teachers. Nine of the 
principals from the BTO schools indicated that high-quality teachers were the driving force 
behind the school’s success, often placing emphasis on providing their teachers with the training 
and support needed to be successful. Most of these schools reported that they had a high or 
moderate degree of control over the hiring of teachers, and all of them indicated that they are 
highly effective in removing teachers and staff that are not meeting expectations. 

Implementation of a particular curriculum and/or instructional technique was the primary driving 
force cited by four of the principals from the BTO schools, and cited among most of the 
interviewed schools as a contributing factor for success. These schools tended to focus on 
effective implementation of their curriculum with curriculum guides, data-driven decisions 
regarding instruction, and programs and/or interventions that complemented the core curriculum. 
Most of these schools also mentioned high-quality teachers as an important component of their 
strategy. 

Coherent instruction was cited by five of the BTO school’s principals as the most important 
driving force for success. However, most of the successful schools cited coherent instruction as 
part of their overall plan for academic accomplishment. Some of these schools place emphasis in 
having a pacing plan that helps teachers know what they should be teaching. Other principals 
mentioned that teacher collaboration as well as the implementation of a particular curriculum has 
helped to strength the instructional coherence at their school. 

An important contribution of the theoretical framework is its representation of how all the 
components relate to one another and ultimately contribute to high academic achievement. In the 
following sections we will elaborate on the different components of the theoretical framework 
through specific examples that were mentioned by the principals. We will first provide examples 
and a detailed description of the three major factors for success: 1) high-quality teachers, 2) 
standards-based curriculum, and 3) coherent instruction. We will then describe the critical roles 
that the other components of the framework (the white ovals in the exhibit) played in ensuring 
the school’s success. We will also provide a description of the roles played by the district, school 
leadership, and external consultants at these schools. The chapter finishes with a presentation of 
the challenges these schools face and a discussion of whether more resources are needed to reach 
an API score of 800 by 2013. 
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Major Factors for Success 

There is a close relationship between the three major factors: high-quality teachers, the 
curriculum, and instruction. High-quality teachers are responsible for delivering the content of 
the curriculum to their classroom. They are held accountable for ensuring that the students 
understand the main content areas that are covered in the curriculum. In turn, the curriculum is 
used to inform the instruction. Finally, the instruction is the way in which the curriculum is 
delivered to the students. These three components all contribute toward high student 
performance. 

High-Quality Teachers 

When asked about the primary strategy for their school’s success, nine high-performing 
principals indicated that having high-quality teachers was the driving force behind their school’s 
success. The other nine BTO schools’ principals also emphasized the importance of teacher 
quality to their school’s success. There are many different dimensions in which one can measure 
the quality of teachers, each of which will be discussed in more detail below. 

Experience, Qualifications, and Education of Staff 

For the most part, interviewed principals did not emphasize the experience or education of their 
staff. However, the principal of Pheasant Elementary School^^ did mentioned that she tries to 
hire new teachers that are trained at a program run through the nearby university because the 
program has produced high-quality teachers in the past. Other interviewed schools appeared to 
benefit when well-trained teachers remained at the school for long periods of time. Puffin, Crow 
and Swan Elementary Schools all indicated that the longevity of their staff (both teachers and 
instructional coaches) was critical to their school’s success. 

Dedicated and Collegial Teachers with High Morale 

The principal of Cardinal Elementary praised her teachers by saying, “I have teachers with heart. 
They go above and beyond the teaching. They make the school a priority and have a vision of the 
future.” There are many ways in which teachers were said to demonstrate their dedication to the 
school and to the students. Eor example, teachers at Dove Elementary put in extra time outside of 
school to working on and developing their lessons. On their own initiative, teachers at Kingfisher 
Elementary sought out new instructional strategies via college courses, books and the internet in 
order to train themselves to be better teachers. As another example, the principal of Hawk 
Elementary said, “Our teachers are very committed and very passionate about these kids. We 
learn what is going on in the family and don’t just look at the child as a student that is here for a 
few hours; we look at the whole child. If the teacher finds that something is going on at home, 
they are on top of it.” 

High Expectations of Teachers 

A number of the interviewed principals said they encourage the dedication and effectiveness of 
their teaching staff by having high expectations for them and making sure that these expectations 
are known. One principal said she shows the teaching candidate a list of expectations at the 
beginning of the interview and does not continue the interview if the candidate does not agree to 
meet all those expectations. The principal of Egret high remarks, “Teachers have to believe and 



All of the school names in this chapter have been changed. 
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breathe the school’s expectations. At Hawk Elementary, the expectations for new teachers are set 
by the veteran teachers, who make it a point to enforce those expectations.” According to the 
principal, “If a teacher is not committed, the other teachers would do something about it. They 
would run that teacher out of the building!” 

Strong Leaders among Teaching Staff 

Some of the principals said they have teachers who take on strong leadership roles. “I want [the 
teachers] to make decisions on things that directly affect them. I just facilitate.” The most 
common way in which schools allow teachers to participate in the decision-making process is 
through a leadership team. At Toucan Middle School, a BTO school, the leadership team goes to 
the district once a month and brings what it learns back to the staff. At Owl Middle School, a 
comparison school, the leadership team works with the school site council to make formal 
decisions regarding the school. In all of these cases, the leadership team is intended to cultivate 
leaders among the teaching staff who will participate in the decision-making processes of the 
school. 

Standards-Based Curriculum 

Since the state’s adoption of academic content standards in 1998, schools have increased their 
focus on implementing a standards-based curriculum. In 2002, the state transitioned to a 
standards-based assessment system, creating even more incentives for schools to align their 
curriculum to the standards. 

Every school that mentioned the specific curriculum they use reported using either the Houghton 
Mifflin or Open Court curriculum, which are both based on the California state standards. Often 
the district will choose which curriculum the schools will use, or give the schools a choice 
between two different curricula. Several principals mentioned that they are happy to use either of 
these curricula because they focus on the same standards they are held accountable for during 
testing. Only one principal expressed mixed feelings: “I welcome the standards and they’ve done 
a great deal to help us work at a higher level. They’ve also hindered me because the district 
wants everybody to work at the grade level, so teachers are hesitant to work above grade level.” 
Below we will provide more details regarding how the specific curriculum interacts with other 
factors to ensure high performance at the school. 

Coherent Instruction 

Instructional approaches among these BTO schools appeared to differ in terms of how much time 
teachers spent on each subject, the extent to which teachers focused on the standards, and how 
teachers addressed the different needs of the students in the classroom. Below we discuss these 
different dimensions of classroom instruction. 

Instruction Tied to Goals, Standards and Curricula 

An important component of instructional coherence is having a schoolwide vision or set of 
common goals related to instruction (O’Day & Bitter, 2003). The most common schoolwide 
vision with respect to instruction that was reported during the interviews was a focus on the 
California state standards. Williams, Kirst, & Haertel (2005) found that the existence of 
coherent, standards-based instruction was a distinguishing characteristic between high and low- 
performing elementary schools in California. An example of a school that has a schoolwide 
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focus on standards based instruction is Dove Elementary, a BTO school, whose principal 
commented, “The teachers are taught that the standards are their bible, and they must teach their 
students the standards because that’s what they are going to be tested on and that’s what they are 
accountable for.” 

Consistency within Grade 

One method of ensuring consistency between same-grade classrooms is by having a pacing plan 
that helps the teachers know what they should be teaching on any given day. The principal of 
Pheasant Elementary, a BTO school, commented, “Every teacher follows a pacing plan so that 
everyone is on the same page. This way nobody goofs off. Before it was Ta la’ land, but now 
instruction is much more structured.” The principals of several other schools agreed that having 
all the teachers on the same page helps ensure that teachers cover all the standards during the 
school year. 

Freedom for Teachers to Try New Ideas within the System 

There are some schools that do not insist on having all the teachers deliver the same instruction 
in each grade-level classroom but still achieve good results. The principal of Sparrow 
Elementary, a BTO elementary school, allows his teams a lot of flexibility to deliver the 
instruction in the way they see most appropriate, as long as they deliver high results. The 
principal explained, “When I say I give the teachers a lot of freedom, I mean that I am not a 
policeman. I don’t go in and get on their case about what they are doing. I am more concerned 
about the effectiveness of what they are doing. I don’t require lesson plans from my teachers, for 
example.” Similarly, the principal of Hawk Elementary says that she gives the teachers the 
freedom to teach how they want, as long as they cover the state standards. 

Differentiating Instruction Based on Student Performance 

One challenge to school success mentioned by several principals was the diverse needs of the 
students in the school and the ability of the teachers to meet those needs. An underlying 
implication of this challenge is that teachers need to be able to differentiate instruction based on 
the needs of their students. The principal of Raven Elementary School commented that the 
school’s teachers spent a lot of time looking at assessment data so that they could differentiate 
the instruction based on what the students knew. According to him, “differentiation of instruction 
is the only key.” Eurthermore, the principal of Toucan Middle School (a BTO school) mentioned 
that differentiating the instruction was the next direction they wanted to take their school in order 
to try to reach an API of 800. 



Other Factors for Success 

As shown in the theoretical framework diagram, at the top is teacher support and training, which 
refers to any professional development opportunities or support that increase teacher capacity. 
Another component that is critical to ensuring high-quality teachers is the hiring and firing 
practices of the school. Control over hiring ensures that the new teachers have the training and 
skill set necessary to meet the expectations of the school. Effectiveness in removing teachers 
gives principals the ability to replace teachers that are consistently unable to meet expectations. 
The combination of hiring well-trained new teachers and removing ineffective teachers 
contributes to a high-quality teaching staff. Einally, teacher collaboration time is used to 
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improve the quality of teachers by allowing them to discuss the curriculum, plan out their 
instructional strategies, and be aligned in their instructional practices and vision for the school. 
Teacher collaboration time also serves as a forum for teachers to look at assessment data 
together. Assessment data is used to inform instruction because it allows teachers to identify 
areas in which students are still struggling and areas which the students have mastered. The data 
can also be used to identify which students need additional help or to group students in their 
classes. 

The three other components of the framework are all related directly to the students. 

Interventions and/or student services can either be additional help to students that are struggling 
or services that are intended to improve achievement. Parental involvement includes students 
receiving extra encouragement from home and parents being informed about the education of 
their students. Finally, high expectations mean that students are encouraged to challenge 
themselves academically and feel confident in their ability to perform at a high level. 

Teacher Support and Training 

Of the complex factors contributing to the individual and organizational aspects of instructional 
capacity, Cohen and Ball (1999) suggest that opportunities for professional and organizational 
learning are among the most crucial. In these interviews, several forms of professional 
development were emphasized, ranging from conferences and workshops to the use of 
instructional coaches and principal and peer classroom observations. 

Training for Teachers that Is Linked to the Standards, Curriculum, Instructional Strategies, and 
Data Use 

Almost all of the BTO schools mentioned providing training for teachers on a wide range of 
topics, including the California state standards, a specific curriculum package, effective 
instructional strategies, using assessment data for instruction, and other non-instructional issues. 
These types of training opportunities have been linked with academic success in several studies 
(Oberman et ah, 2005; Bitter, Perez, Parrish, Gonzalez, Socias, Salzfass, et ah, 2005); however, a 
study by Williams, Kirst, & Haertel (2005) found that the provision of professional development 
opportunities, although positively correlated with academic success, was not statistically 
significant. 

Specific examples of professional development opportunities that were mentioned by BTO 
schools were training in Guided Language Acquisition Development (GLAD) to learn strategies 
for vocabulary development and grammar for ELs, sending teachers to the AVID (Advancement 
Via Individual Determination) institute to learn how to implement the AVID program in their 
school, training in Ruby Paine’s work on understanding poverty, quantum learning (which looks 
at how students learn), and training in instructional strategies for the Open Court curriculum. 
Furthermore, many schools received state funds through Assembly Bill 466 to provide training in 
reading and math to teachers, paraprofessionals, and instructional aides. 

These professional development opportunities can be provided either in house, through the 
district, or through an external organization. Schools such as Crow Elementary determine the 



The text of Assembly Bill 466 can be found at http://www.leginfo.ca.gov/pub/01-02/bill/asm/ab_0451- 
0500/ab_466_bill_2001 101 l_chaptered.html 
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type of professional development that the teachers would benefit from through a professional 
development team. This team meets once a week and consists of the literacy and math coaches, 
bilingual advisor, principal, assistant principal, and SB 65 coordinator. This principal 
emphasized the importance of offering professional development opportunities that “really make 
sense and the teachers will buy into. [It is also extremely important that] teachers have the 
chance to use it, not just as a one time shot, but really having a lot of consistency and 
continuously going back to it over the year.” 

Instructional Coaches 

Instructional coaches were said to help teachers in a myriad of ways, such as teaching them 
instructional strategies for the classroom and how to use assessment data, taking over a teacher’s 
classroom so that the teacher can observe other classrooms, and giving demonstration lessons for 
the teachers. The most common type of instructional coaches that were mentioned by schools are 
literacy and math coaches, although one school mentioned have a science coach as well. These 
coaches are often provided by the district and have been observed in high-performing schools in 
several research studies (Oberman et ak, 2005). 

The instructional coaches can also play important leadership roles in the school. The principal of 
Crow Elementary School remarked, “I have to attribute part of the [school’s] success to the 
leadership of the coaches, who have been here for four years. That has been great for 
consistency.” 

Classroom Visits 

One strategy observed in high achieving high schools by Oberman et al. (2005) was that the 
principal would visit the classrooms on a regular basis to monitor the progress of the students 
and increase accountability of the teachers. This practice was also mentioned by four of the 
interviewed principals. They mentioned many reasons for visiting the classrooms, including 
assuring that the students were engaged, supporting the teachers, identifying which students were 
having trouble, and monitoring whether the common curriculum was being implemented 
correctly. The principal of Albatross High School emphasized the importance of classroom visits 
for accountability by saying, “When I became principal, I told teachers that they were going to 
be visited, so they better get used to it. I want our superintendent in our school. I want teachers 
from the school that is half a block away to come to our school and see us as a model for what a 
successful school should look like. Congressmen, senators, I encourage everybody to come by 
and walk through our school. It makes teachers more accountable when they know that 
somebody is going to come visit them.” 

Peer Coaching and Mentoring 

Peer coaching and mentoring allow teachers to learn from each other in either an informal or 
formal setting. Principals mentioned several different ways in which they organized peer 
coaching and mentoring. Crow Elementary School pairs struggling teachers with teachers that 
are effective in the areas that are troublesome for the struggling teacher. Other schools target new 
teachers for peer coaching and mentoring. Eor example. Dove Elementary uses the Beginning 
Teacher Support (BTS) system, in which each new teacher is assigned to an experienced teacher, 
with whom they meet periodically throughout the year. They have criteria that they have to meet, 
and they do write-ups of how they are progressing on these criteria. The principal of Raven 
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Elementary commented, “Knowing there is someone out there who the new teacher can turn to in 
a moment of need is very important. Nobody feels isolated.” 

Peer Observations 

Peer observations are an opportunity for teachers to observe the teaching practices of their 
colleagues in other classrooms. Some schools have peer observations on an “as needed” basis; 
for example, only struggling teachers may be asked to observe another teacher, or peer 
observations may only take place when they are introducing a new strategy. Other schools have 
regularly scheduled peer observations, such as Toucan Middle School, which has three days in 
which teachers have the opportunity to see what other teachers are doing in their classrooms. 
Another example is Albatross High School, which has a group of four or five teachers observe 
classrooms twice a year and are asked to observe something specific in each classroom (such as 
the environment of the classroom or instructional strategy). Furthermore, peer observations at 
Albatross High School are inter-disciplinary exercises in which teachers are expected to visit 
classrooms outside of their subject area. The principal of Albatross High school supports the 
system, saying, “We are trying to develop a system where teachers have more accountability to 
their peers, rather than a principal versus teacher type of thing.” 

Conferences and Workshops 

Conferences and workshops are intended to provide teachers with a forum to come together and 
learn about issues relevant to the field of education. One principal mentioned sending teachers to 
conferences and writing workshops; however, the teachers must attend these on their own time. 
An alternative to sending the teachers to external conferences and workshops is to have staff 
presentations that are organized in a conference-like setting. Kingfisher Elementary, a BTO 
school, dedicates two days to having teachers with expertise in different areas present to their 
peers. The principal of Kingfisher Elementary remarks, “We bring in very few outside experts 
for professional development. Instead we use our own expertise.” 

Hiring and Firing Practices 

Control over Hiring Staff 

Schools with high degrees of control over the hiring process typically brought candidates to the 
school to be interviewed by the principal, the department chair, and a student. After these 
interviews, the principal would decide who to hire. Schools with less control over hiring included 
one that was able to choose its teachers at the beginning of the summer, but was forced to hire 
whoever the district had available based on credentials and seniority if an opening arose at the 
end of summer. Pelican Middle School, a comparison school, is told it they must hire by the 
district at any time of the year. One BTO school’s principal said they face a similar challenge: 
“[Not having a choice in who I hire] affects my ability to change the culture of the school. So I 
feel that my hands are tied.” 

Effectiveness in Removing Teachers 

The firing practices of a school are complicated by the restrictions on removing teachers. Given 
that very few schools have complete control over firing teachers, principals said they tended to 
focus on removing teachers that were clearly not meeting school expectations. The most 
common way these principals said they were able to remove teachers was by encouraging them 
to leave by themselves. The principal of Egret high, a BTO school, said, “We have certain 
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expectations for teachers that are not negotiable. If a teacher is not getting there, I do everything 
I can to make that teacher miserable. My rule is that if I would not put my child in that class, I 
would not put your child in that class. Parents entrust me to provide a good education.” On the 
other end of the spectrum are schools that are not effective in firing because they have been 
unable to, or have made little attempt to, remove ineffective teachers. The principal at Falcon 
Elementary, a comparison school, said that at his school, “Bad teachers don’t move on. I hope 
they leave.” When another comparison school’s principal was asked if there was high turnover in 
the teaching staff, she responded, “There’s no turnover, and that is part of the problem!” 

Most of the interviewed principals said that their first response is to try and help their struggling 
teachers to improve. The principals of Toucan Middle and Swan Elementary Schools mentioned 
having a peer assistance and review (PAR) program, which puts together a team of 
representatives from the district, the union and the school to come up with a plan for struggling 
teachers. “Our goal is not necessarily to remove the teacher, although we’ll do that if it is 
necessary. The goal is to help the teacher get better.” 

Exhibit 5.2 categorizes the interviewed schools based on descriptions of their hiring and firing 
practices. All but three of the BTO schools’ principals said they have a high or fairly high degree 
of control over hiring and firing. While two of the comparison schools also said they had a high 
or fairly high control over hiring or removing teachers, the other three comparison sites said they 
had a lower degree of control. 



Exhibit 5.2. Hiring And Firing Practices In BTO And Comparison Schools 



Degree of Control 


BTO Schools 


Comparison Schools 


over Hiring 


Count 


% 


Count 


% 


High 


4 


22% 


1 


20% 


Fairly high 


11 


61% 


1 


20% 


Medium 


3 


17% 


1 


20% 


Low 


0 


0% 


1 


20% 


None 


0 


0% 


1 


20% 


Total 


18 


1 00% 


5 


100% 



Teacher Collaboration 

Regularly Scheduled Collaboration Time 

Almost every interviewed principal mentioned that they have collaboration time for teachers. 
These findings are consistent with Williams, Kirst, & Haertel (2005), who did not find that 
encouraging teacher collaboration was a distinguishing characteristic between schools with high 
and low student achievement. 

Although nearly every school had regularly scheduled collaboration time, the variation was in 
how they spent the time. Some of the commonly mentioned activities were looking at assessment 
data, coming up with instructional strategies, designing lesson plans, identifying students that 
may need additional help, and receiving professional development. Often the instructional 
coaches attended the teacher meetings to help lead the discussion or provide professional 
development. 
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Use of Assessment Data 

California public schools have engaged in numerous federal, state, district, and school-mandated 
assessments to monitor student progress. Data from these tests ean be invaluable to schools by 
assessing the needs of students and pointing to areas for improvement (Neumann, 1996; 

Kannapel and Clements, 2005). Williams, Kirst, & Haertel (2005) found using assessment data 
to improve student achievement and instruction to be positively correlated with high 
performance. For some schools, the data drives their entire strategy for success. For example, the 
principal of Flamingo high (a BTO school) proudly said, “I’m known as the data principal. I look 
at the data. I know my data. And I act on my data. We tear the data apart.” 

Use of Several Assessment Measures 

There are numerous different assessments that can be used by schools. Some schools use the 
assessments associated with Open Court and Houghton-Mifflin, which can be administered on a 
six-week or quarterly basis. In addition, some schools created their own assessments that aligned 
with state standards or used district-developed benchmark assessments. State assessments, 
ineluding standardized tests such as the CST, CAT/6, CELDT, and APRENDA, were also 
mentioned as commonly used assessments. High schools also have the California High School 
Exit Exam (CAHSEE) when students are in the tenth grade, which can also be used to evaluate 
how students are performing. Most schools used a combination of these assessments to inform 
their instruction. Eor example. Albatross high, a BTO school, used a combination of the CST, 
CAHSEE, ABC (assessment of basic content standards) tests and other common assessments 
given throughout the year. 

Tests Aligned With the Curricula and Standards 

Almost all of the interviewed schools mentioned using the CST and/or CAT/6, which are 
standardized tests that are aligned with the California state standards. The next most commonly 
mentioned assessments were the tests that accompany the Open Court and Houghton-Mifflin, 
whieh are also completely aligned with these curricula. Schools that have their own school 
benchmarks also seem to make an effort to align the tests with the standards. Ealcon Elementary, 
Toucan Middle and Jaeger High School all commented that the school benchmarks were based 
on the content areas and were designed to evaluate whether students were learning the California 
state standards. 

Data Used to Monitor Student Performance, Inform Instruction, and Monitor Practice of 
Teachers 

The use of assessment data to monitor student performance was reported in several different 
ways. The principal of Kingfisher Elementary, a BTO school, reported using the CST and 
demographic data to create a comprehensive picture of each student. According to him, “It’s 
important to have the ability to collect and analyze data for each and ever child by name, so that 
we know exactly (in a prescriptive sense) what instruction each student needs.” In addition, more 
frequently administered tests, such as the Open Court or Houghton-Mifflin tests or district 
benchmarks, were used to monitor student performance on a more regular basis. Eor example. 
Dove Elementary and Swan Elementary (both BTO schools) both mentioned that their teachers 
reviewed the assessment data every week during their teacher collaboration meetings to identify 
the areas in which students were struggling and excelling. At Toucan Middle School, a BTO 
school, the teachers and students reviewed the school benchmark results together. The principal 
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remarked, “It is really interesting to see the ownership of achievement passed from our teachers 
onto the students.” 

In addition to monitoring the performance of the students, some schools use assessment data to 
inform instruction. Teachers at Swift Elementary School, a BTO school, used the results of the 
Open Court assessment, which is administered approximately every six weeks, to plan for the 
next unit of the curriculum and identify areas from the last unit that they needed to review. Data 
also reveal which instructional strategies are working better than others. For example, some 
principals reported asking teachers whose students had performed well on the assessments to 
share with the other teachers what they had been doing in the classroom. 

Another common use of assessment data is to evaluate and monitor the effectiveness of teachers. 
“Using the data has given teachers the opportunity to look at something objective and take 
responsibility for both the success and the problems.” After identifying teachers who need help, 
the principal will work with them to understand why their students are performing at such a low 
level and what they can do to improve. However, there was clearly hesitancy on the part of some 
principals to single out teachers who had poor test score data. The principal of Falcon 
Elementary, a comparison school, says that the teachers would start screaming if he started 
singling out particular teachers, so instead he gave tips for improvement to all the teachers during 
the grade-level meetings, instead of identifying teachers having problems. The principal of Egret 
high described his attitude towards this issue by saying, “The data tells us how we are doing, 
where we are successful, where we are failing. It generates the opportunity to compare 
classrooms and promote teacher conversations. Data are not threatening. We don’t use data to 
beat up the teachers. We use data to improve.” 

Assessment Data Frequently Reviewed 

Schools vary in the frequency of the assessments they administer and how often they review 
them. Many schools mentioned assessing the students through school benchmarks or the 
curriculum tests approximately every four to six weeks. More frequent assessments were 
reported at schools such as Dove Elementary (a BTO school), which administered a mini- 
assessment for their students every week, with the teachers meeting on Wednesdays to review 
those results. An even more extreme case is Egret high (a BTO school), which assessed its 
students through tests and quizzes on a daily basis. On the other end of the spectrum is Owl 
Middle School (a comparison school), who only mentioned using the CST for assessment data, 
which is administered once a year. 

Teachers Teaching the Same Course Giving the Same Tests 

A study of best practices in high achieving high schools (Oberman et ah, 2005) described 
advantages of having all the teachers of the same grade level of subject administer the same tests. 
Most of the principals interviewed in this study gave the impression that the assessments they 
used were uniformly administered in their school. For example, all schools that mentioned using 
the state standardized tests (such as the CST, CAT/6, APRENDA, and CELDT) or the 
assessments associated with the curriculum (such as Open Court of Houghton-Mifflin) 
administered these tests to all the classrooms. As a result, they were able to compare the 
performance of each classroom and student over time. Other schools, such as Toucan Middle and 
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Dove Elementary (both BTO schools), had developed school benchmarks or mini-assessments 
that were also administered in every classroom on a regular basis. 

Consultant or Coach Helps Teachers Interpret Data 

One common function of the instructional coaches is to help teachers understand how to use 
assessment data. Schools such as Swift Elementary and Parrot Elementary (both BTO schools) 
dedicate some time during teacher collaboration meetings to having the instructional coaches 
provide guidance to the teachers about what to focus on when looking at the data, and how to 
improve their instruction based on what they observe in the data. Instead of instructional 
coaches. Albatross high hired an outside consultant to teach teachers how to use the data the 
understand student achievement. According to the principal, this has really helped the teachers 
use the data to improve instruction and achievement. 

Use of a User-Friendly Data Monitoring System 

Data monitoring systems were observed in high- achieving high schools by Oberman et al. (2005) 
and were also mentioned by several of the principals interviewed for this study. Two schools 
mentioned using Edusoft, which allows them to break down their benchmark tests. In addition to 
Edusoft, Kingfisher Elementary also uses the Data Warehouse, which is provided by the 
county’s department of education. These programs allow the teachers to bring together multiple 
sources of data. Toucan Middle, a BTO school, uses a different program called the Data 
Director, which was provided by the district and seems to perform a function similar to Edusoft. 
The last data monitoring program that was mentioned was Pulleyman, which is used by 
Albatross High to break down data for the teachers. 

Intervention and/or Student Services 

The Oberman et al. study of best practices (2005) found that successful schools focus on timely 
interventions for students that are struggling and need additional academic help. The following 
are short descriptions of different types of interventions and student services described by some 
of the principals for this study. 

Tutoring Programs 

The most commonly mentioned interventions are tutoring programs held outside of regular 
school hours. These programs can be taught by teachers at the school or by instructional aides 
who provide extra help to those students having difficulty understanding the material that is 
taught during the day. Pelican Middle School offers their tutoring program during lunchtime. 
They hire certified teachers (called clinicians) to come in and offer interventions during 
lunchtime for struggling students. 

Similar to the after school tutoring programs are Saturday intervention classes, which were 
mentioned by two BTO schools. According to the principal of Swift Elementary, “Saturday 
intervention is more successful than after school because the teachers and the kids are really tired 
after school. If they are not performing during school, they are not going to do well after school.” 

Another option for tutoring programs outside of the regular school day is summer school or 
inter-session classes for schools on the year-round schedule. Pheasant Elementary has inter- 
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session intervention for struggling students, especially ELs. These classes are for 20 days and are 
four hours per day. 

Targeted Educational Programs 

Some BTO schools have implemented targeted educational programs such as AVID, GATE, 
Reading Recovery, Read 180 and Read Across America. Half of the high-performing high 
schools we interviewed had implemented AVID in their schools, which is a program that focuses 
on preparing students for college. The principal of Albatross high said, “We went to a week-long 
AVID seminar and decided to give the strategies to everybody.” In addition to AVID, Albatross 
high also has a schoolwide writing program that occurs in every classroom in the school for two 
days of the year. 

Reading Recovery, Read 180 and Read Across America are programs that focus on improving 
students’ reading abilities. Puffin Elementary, a BTO school, has implemented Reading 
Recovery for the past 12 years. Swift Elementary, a BTO school, participates in Read Across 
America, which is a reading awareness program that encourages students to celebrate reading on 
a particular day of any given year. 

Counseling or Other Student Services 

Many students have issues outside of school that interfere with their ability to succeed 
academically. Accordingly, Swift Elementary School has four counselors that come to the school 
twice a week. The principal commented, “I believe in counseling because these kids come to 
school with so many issues. That’s why they are not successful.” Most schools have the 
counselors come in a few times per week, although it was mentioned by a few schools that 
having a part-time counselor was not sufficient. There are other programs as well that are 
designed to improve student behavior. Crow Elementary uses the “Motivation and Maintenance 
Dropout Prevention Program,” which has a coordinator that calls the parents (and offers 
assistance if appropriate) if a student is not at school. Egret High uses “Capturing Children’s 
Hearts,” which is a toolbox for students to become self managing. The principal reported 
noticing a decrease in discipline problems and an increase in student achievement since 
implementing this program. 

Parental Involvement 

There has been extensive research dedicated to the effects of parental involvement in schools 
(Eullan, 1991; Levine & Lezotte, 1990; Purkey & Smith, 1983). The lack of parental 
involvement was a commonly mentioned challenge for schools, but research by Williams, Kirst, 
& Haertel (2005) did not find that involving and supporting parents was correlated with high 
student achievement. Nonetheless, some interviewed principals felt that parental involvement 
was key to the success of their students. The principal of Swift Elementary commented, “The 
kids that have parents who support school are the ones that are going to do well.” 

Existence of Activities to Engage Parents and Families 

Most schools hold events, such as parent-teacher conferences, open house, parent night and 
back-to-school night. These events give parents an opportunity to discuss the progress of their 
children with their teachers and other school staff. In addition. Toucan Middle School has a 
college night, where the students present their college plans to their parents. 



American Institutes for Research 



Page 75 




Successful California Schools in the Context of Educational Adequacy 



Some schools go a step further by having regularly scheduled meetings with parents. Pheasant 
Elementary has a weekly workshop for parents whose children are struggling where they discuss 
how the parents can help their child at home. Similarly, Sparrow Elementary has a morning 
meeting with all parents every two weeks in which the teachers (and a translator) tell them what 
they need to do for the students to be successful. According to the principal, “Parents want to do 
the right thing, so if you train them while their kids are in kindergarten, then they come back in 
later years. They want to renew the dream. They like the vision of their child in a cap and gown. 
My advice to other principals is to meet with parents when their children are young.” 

Parents can also get involved with the school through booster clubs, the school site council, and 
volunteer opportunities. A core group of parents at Dove Elementary helps with the 
photocopying at the school and running the school bank. Eurthermore, they are starting a 
program called “P for 3” where every parent is expected to commit three hours to the school 
each year. 

Swift Elementary and Hawk Elementary both mentioned having a center where parents can take 
classes and get information on how to get involved with their child’s education. Eor example, the 
parent center at Swift has ESL classes two days a week and computer classes two days a week 
that are taught by teachers from the nearby high school. The parent center at Hawk offers 
counseling services for the parents. 

Good Communication between Parents and School 

One challenge to having good communication between a school and its parents is that many 
parents are not English speakers. Albatross high, a BTO school, addressed this problem by hiring 
Vietnamese and Spanish-speaking community liaisons. These liaisons come to parent night and 
explain to the parents about the school, standardized testing, and ways in which they can help 
their students. Since they hired the community liaisons, participation in these parent nights has 
grown appreciably. Another example of improved communication is Jaeger High, where they 
purchased a system on the internet called NTI that sends phone messages in both English and 
Spanish to all parents about testing, half days at school, and other relevant information. Einally, 
the previously mentioned Motivation and Maintenance Dropout Program at Crow Elementary 
has a coordinator contact the parents of children who are absent; the coordinator is available to 
work with those parents to solve any serious problems. All of these methods are intended to 
break down the communication barriers between the school and parents. 

High Expectations of Students 

“High expectations” has been a mantra of the school reform movement since the late 1980s. 
Williams, Kirst, & Haertel (2005) found a positive correlation between setting clear, high, and 
measurable expectations for student achievement and high performance. However, they found no 
correlation between high expectations for student behavior and high performance. In this study, 
the following examples of school expectations were described. 

School Sets High and Measurable Expectations for Student Achievement 
What constitutes high expectations is a matter of debate. The principal of Sparrow Elementary 
commented, “The fallacy with the word ‘high expectations’ is that nobody believes that they 
have low expectations, but it’s just because they have never seen a high expectation. How can 
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teachers know what a high expectation is until they’ve seen a teacher that is doing better than 
them [sic].” The principals of two schools, Flamingo High and Toucan Middle School, said they 
express their high expectations for students by encouraging them to take challenging classes. At 
Flamingo High, the principal designs the class schedule of all the students in order to take 
higher-level math. Before he did this, the principal said that the students were not selecting any 
advanced classes. The principal of Toucan Middle School remarks, “We provide rigorous 
courses, and we believe that students can achieve at that level. All our seventh graders are in 
algebra, and we really believe that the students are capable of meeting that rigor and our test 
scores show that they are.” 

Other schools express their expectations of students through verbal encouragement. The 
principal of Pheasant Elementary School believed that high expectations are the main reason 
why her school was outperforming all the other schools in the district. She explained, “Every 
morning, I have an assembly where I repeat the same things over and over. Eor example, I shout 
out, ‘What’s our API score?’ and they all say, ‘800 or more!’ Before the students take a test, I 
come to the class to talk to them and motivate them. Expectations are even more important than 
the curriculum or the materials used.” A similar strategy was reported by the principal of 
Sparrow Elementary, another BTO school, who shouts during the morning assemblies, “Which 
school has the smartest kids? Which school has the best teachers?” and the students shout, 
“Sparrow!” This type of verbal encouragement was touted as very effective for these two 
schools. 

Policies for Student Behavior 

Dove Elementary has a formalized procedure for their expectations for student behavior through 
their “Eagle Expectations.” These are in the student handbook and are also shared with the 
parents. To make sure that all students know and understand the Eagle Expectations, the 
principal and assistant principal go to all the classrooms to talk about them. They also go over 
the expectations with the new students at the beginning of the year. 

Another strategy is to focus on expectations for school attendance. Eor example. Crow 
Elementary’s Motivation and Maintenance Dropout Program calls parents every time their child 
is not at school. Pheasant Elementary has an incentive system where students can earn “pheasant 
dollars” for each day they come to school. With these dollars, the students can buy things from 
the school store. 

School Climate or Culture 

Some principals mentioned their emphasis on making the school a fun and exciting place for the 
students to be. The principal of Swift Elementary School commented, “We’re always finding fun 
things for the kids to do. These days the academics are so difficult and they are tested all the 
time. So we’re trying to keep it fun and exciting for them.” Her school offers music, orchestra, 
student council, an art class, and an after-school program. They also have a career day and a big 
party for the students that is given by a community organization. Similarly, the principal of a 
comparison school placed the same importance on making the school a fun place. She said, “We 
need to spice things up and become a little more entertaining. School is boring for kids — at 
home, they are on their computers and playing video games.” As a result, the principal of this 
school is trying to use more technology to get students more excited about learning. 
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School, District, and External Leadership 

School and district leaders and external consultants all have the potential to influence school 
performance. While only a few schools have hired an external consultant, every school has an 
identifiable leadership structure and a relationship with the district that ranges from minimal 
interaction to close and collaborative. The next three sections briefly describe these three sources 
of leadership as described by the study respondents. 

School Leadership 

Leadership has long been considered one of the most important contributors to effective schools 
(Davis & Thomas, 1989; Purkey & Smith, 1983; Terry, 1996, Muijs, Harris, Chapman, Stoll, and 
Russ, 2004). Effective leadership can be eritical for school improvement efforts, particularly 
because their contribution to the school cannot be measured or re-created the way one could with 
the other components of the framework. 

Leadership Structure 

The most common leadership structure is centered on a dominant principal, seen in 1 1 of the 18 
BTO schools included in this study. As stated by the principal of Sparrow Elementary, “There 
has to be somebody on the campus with that passion and drive; somebody who has a belief in the 
kids.” The principals took on leadership roles in various ways. Some of them played a critical 
role in establishing the expectations of the students by developing a student behavior handbook 
(as done at Dove Elementary) or deciding the class schedule for every student (as at Elamingo 
high). Another role of strong principals is to set the expectations for staff members, providing 
them with the guidance, support and training they need to be successful, and effeetively dealing 
with those not meeting expectations. Einally, principals who play a strong leadership role often 
have a large say in the curriculum and instruction that is delivered in the school. The principal of 
Swan Elementary School dedicated a lot of time to finding supplementary materials for her EEs 
when the school switched from dual immersion to English Only. Similarly, the first thing the 
principal of Sparrow Elementary did when he arrived to the sehool was visit every classroom and 
determine what kind of textbook and materials they should be using. 

Beyond a single dominant leader, Elmore (2000) suggests that “the guidance and direction of 
instructional improvement” (Elmore 2000) requires the distribution of leadership among staff. 
Elmore suggests that as leadership becomes more distributed, the quality of leadership will 
evolve by enhancing the skills and knowledge of staff, creating a common culture of 
expectations, fostering productive relationships, and “holding individuals accountable for their 
contributions to the collective result.” Eour of the high-performing principals indicated this type 
of shared leadership between the staff and the principal. The principal of Pelican Middle School 
commented, “I believe in shared leadership. I think a lot can happen if a teacher buys into what 
they do. And it is much easier if the new idea came from a teacher.” 

At the other end of the school leadership spectrum are schools where the teachers are the primary 
leaders, with support from the principal, as reported in two BTO schools. The principal of Hawk 
Elementary School remarked, “I am not the ‘principal’; I am just there to help and support the 
teachers. I facilitate their ability to teach.” At Hawk Elementary, the teachers have taken it upon 
themselves to set their expectations and to force out any teacher that is not meeting them. Each 
teacher is free to teach the curriculum in any way they like, as long as they are teaching the 
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standards. The teachers also have taken it upon themselves to organize their own one-on-one 
after school tutoring program for struggling students. The principal of Parrot Elementary School 
takes on similar responsibilities in their school. It is noteworthy that both of these schools have 
fairly new principals, which may be one reason why teachers have stepped into primary 
leadership roles. 

In contrast, the principal of one comparison school reported an unclear leadership structure. He 
remarked being a bit overwhelmed with the responsibilities that he is expected to take on in the 
school. He says, “I don’t have an MBA. I don’t have a degree in psychology to deal with my 
problematic teachers and students. I don’t know about seismic retrofitting. Yet these are things 
I’m supposed to be an expert on.” The teachers at this school also do not appear to be stepping 
into leadership roles. They appear to be isolated in their classrooms and are reluctant to 
participate in teacher collaboration or professional development. According to the principal, the 
primary challenge the school faces is building the instructional capacity and collegiality of the 
teachers. 

Professional Development for the School Leader 

Many districts provide extensive training for the principal and other school leaders. The most 
commonly mentioned training for principals is provided through Assembly bill 75, which 
mandates that all administrators receive training from the district that covers topics such as 
“curricular and instructional leadership, improving the abilities of new administrators to assess 
the performance of teachers, and providing a foundation for the proper implementation and 
examination of all pupil assessments.” The principal of Toucan Middle School, a BTO school, 
referred to this training at the district as “new principals’ academy,” where they cover topics 
such as finance, curriculum, maintenance, and facilities. Some districts have meetings for all the 
principals to discuss their schools’ strategies and leam from other principals’ successes and 
failures. 

District 

In a review of recent research literature on the role of the district in school-level reform efforts, 
Hightower, Marsh, Talbert, & Wechsler (2000) found that districts can play a “potentially critical 
role in improving teaching and learning.” Districts that act as change agents tend to mobili z e 
critical resources (human, social and physical), thus having a “better chance of enacting and 
sustaining state and local reform goals and policies.” The principals of the interviewed BTO 
schools revealed a broad spectrum of levels of district involvement in the management, goals, 
and strategy of the schools. In the following sections, we will discuss examples of how districts 
were involved in establishing goals for the school and providing assistance to the schools for 
accomplishing those goals. We will also give examples of schools in which the district does not 
play an active role in the school’s operations. 

District Establishes District-Wide Goals for the School 

Two schools mentioned that their district has explicit district-wide goals for all their schools. For 
example. Owl Middle School (a comparison school) has two district-wide goals. One is that 100 
percent of their high school students graduate and the second is that at least 26 percent of the 



The text of Assembly Bill 75 can be found at http://www.leginfo.ca.gOv/pub/01-02/bill/asm/ab_0051- 
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American Institutes for Research 



Page 79 




Successful California Schools in the Context of Educational Adequacy 



students score at the proficiency level on the state standardized tests. The principal of Owl 
Middle School has tried to align the school goals with these two district-wide goals, but feels that 
at times the district is too demanding and unrealistic about meeting the district goals. He 
comments, “Sometimes we are put under the microscope by the district and sometimes they give 
us negative feedback. We have such a diverse population and have students with difficult 
circumstances at home. Sometimes the district forgets that.” 

Toucan Middle School also has two district-wide goals: the first is that students are proficient on 
the CST after five years and the second is that ELs are redesignated after four years. The 
principal of Toucan Middle School has a more positive outlook on the district- wide goals than 
the principal of Owl Middle School. She says, “The district and the school are on the same page 
about doing the right thing for our kids.” 

District Plays an Active Role in Achieving School Goals 

The most common way that districts assist schools in achieving their goals is by providing 
professional development for school staff. Nine schools reported receiving some form of 
professional development from the district, and most expressed a favorable opinion of the 
training they had received. Examples of professional development organized by the district are 
principal training through Assembly Bill 75, additional opportunities for school leaders to come 
together to exchange ideas and information, writing programs for teachers, GATE training, and 
specific subject area courses offered to teachers on the weekends, after school, or during the 
summer. 

Besides professional development, the district can get involved by providing the school with 
programs to analyze the data with, curriculum guides, pacing maps, textbooks, and technical 
support. The district also provides the financial support necessary for the school to meet its 
goals. The principal of Jaeger high said that, “If I can’t run a program given the money [I have], 
the district pretty much says, ‘Okay, we’ll help you out on it.’ They’ve always worked out a way 
to fund our programs, even if I have to pay it back later.” 

District Does Not Play an Active Role in School’s Operations 

The principals of five of the BTO schools indicated that their district did not play an active role 
in the school’s operations. Three of those schools seemed to be satisfied with that arrangement. 
The principal of Sparrow Elementary School commented, “The district helps me because they 
leave me alone and let me do my thing.” Swift Elementary and Flamingo High also expressed 
appreciation that the district did not get very involved in the school’s operations, which allowed 
them the freedom to try new strategies. Some schools with high district involvement, such as 
Raven Elementary, indicated that they would prefer for the district to play a less active role in the 
school. 

On the other end of the spectrum are two BTO schools that would like more district support. The 
principal of Albatross high even said that his school had been neglected by the district because 
they did not believe that the school could improve when they were low performing and did not 
support many of the programs that they wanted to implement. 
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External Consultants 

A few principals mentioned hiring external consultants. Toucan Middle hired an independent 
company to develop an audit of the school that focused on instruction and data. The company 
developed the school benchmarks and also helped teachers improve their instructional 
techniques. Dove Elementary is another BTO school that hired external consultants to show the 
teachers how to analyze test data and provide guidance to teachers that are struggling. 



Challenges 

Each principal interviewed for this study was asked to identify the biggest challenges they faced 
in increasing the academic performance of their school. These challenges were collapsed into 
five categories, shown in Exhibit 5.3. The exhibit shows frequencies for the greatest single 
challenge mentioned by the principal at each school. 



Exhibit 5.3. Frequency Rating of Each Category as the Most Chailenging issue* 





Rated as Most Challenging 
Factor by BTO Principals 
(frequency) 


Rated as Most Challenging 
Factor in Comparison 
Schools (frequency) 


Characteristics/needs of student 
population 


8 


3 


Administration/structure 


0 


0 


Teaching 


4 


1 


Finances/resources 


2 


1 


Parental involvement 


2 


0 



* Two beating-the-odds schools did not identify an important challenge to their success. 



Perhaps the major finding from this table is the similarity in the general distribution of responses 
between principals from BTO and comparison schools included in this study. A majority of 
schools in both groups enroll relatively high percentages of challenging students, so it is perhaps 
not surprising that both groups of schools cited the high needs of their students as a major 
obstacle to their success. It is also interesting that only a few respondents from both groups saw 
finances as their major concern. 



Funding Required to Reach an API of 800 

The administrators from the schools we interviewed gave mixed responses regarding whether 
they need increased finances to reach or maintain an API of 800. Eor those schools that were at 
an API greater than 800 (4 schools), half of them said that they had adequate resources to remain 
at the targeted goal. Eor instance, the principal from Mockingbird Elementary School articulated, 
“I think we do have the resources, I think if anything it is just a matter of getting the training and 
convincing the staff that they need to continue to work hard, that is more of a leadership thing, 
the district just throwing money in there will not make a difference.” 

Meanwhile, the other half expressed frustration due to lack of funding. Puffin Elementary 
School’s principal lamented, “We should not have to write grants to maintain programs. . ..It is 
absurd that I am struggling for resources for things like mental health counselors. We are 
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working on a shoe-string budget. . ..We can't fully implement what we want — we have to have 
more resources.” For Hawk Elementary School, the $60,000 cut from their Title 1 funds resulted 
in the elimination of their after-school program. Two years ago, their annual fiscal budget was 
$100,000 more than their current allocation. Their principal remarked that they used to have the 
level of resources necessary to sustain an API score of 800, but that now the cuts are going to 
affect their performance. 

A somewhat different trend is seen with BTO schools whose API scores are below 800 (14 
schools). Of these schools, nine felt that an API of 800 was an obtainable goal for the year 2013; 
the other five felt that the goal of 800 was not obtainable. Of the administrators that saw 800 as 
obtainable, none saw more funding as necessary, but the majority stated that money would be 
helpful in achieving the goal. A few expressed satisfaction with their present level of funding. 
For instance, while the Swift Elementary School’s administrator would like to have more money 
to have more intervention programs, the principal from Pheasant emphasized that their concerns 
do not directly relate to funding: “Funding is not so necessary. We have intervention programs 
for at-risk students — we do that for 80 hours. . . .There is also the inter-session programs. 
However, we are not getting support from parents.” Both principals from Cardinal and Parrot 
Elementary Schools admit that increased funding is beneficial because it can help provide 
additional support programs. Dove Elementary School is one of the few that are content with the 
funding they receive; its principal noted that their Title I funds are sufficient to achieve the 
school’s goals. 

Exhibit 5.4. Attitudes toward Attainability of 800 API Score and Necessity of Additional Funding 
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Among the five BTO schools that were below 800 and whose administrators stated that reaching 
an API of 800 in 2013 was unlikely, only one administrator identified an increase in funding as 
crucial for the changes they envisioned for their school. The principal for the lone exception, 
Jaeger High School, wanted more intervention programs and reading classes for the school’s 
students and this would entail a higher financial allocation. For the rest of the schools, while 
some acknowledged that money is helpful, none felt that more funding was unnecessary. For 
example. Raven Elementary School’s principal asserted, “I wouldn’t say no to the money, but I 
don’t need more money for those changes [strategies for writing improvement and incorporation 
of structured computer use].” Meanwhile, Toucan Middle School’s administrator maintained, 
“Honestly, I don't think that additional funding is necessary. We are fortunate to have the 
funding we have and we spend it wisely. . . .The level of funding we have right now is 
appropriate.” 

Regarding the comparison schools, all principals mentioned that 800 was an obtainable goal. In 
addition, they all also affirmed that having more resources would facilitate the changes they 
would like to implement and two administrators explicitly stated a need for greater funds. For 
instance, the principals from Falcon and Blackbird Elementary Schools both admitted that an 
increase in budget allocations would translate into more supplemental instruction materials and 
support programs. Moreover, when asked whether they needed more funds to achieve the 
changes they envisioned. Hummingbird Elementary School’s administrator responded, “Last 
year, I would have said no, but my funds have shrunk this year because the state has cut back on 
the LEP and EIASCE funds. . ..I would need another $200K for people. We need to hire staff to 
work with the kids who need extra support. That is about 5 or 10 percent of our current budget.” 

In sum, while only a few school administrators explicitly stated their need for additional funds, 
the majority acknowledged that most changes would be better implemented with increased 
finances. 
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Chapter VI. Summary of Results 



This report has analyzed successful California schools in the context of educational adequacy. It 
has covered a range of topics, including how the “successful schools” approach to considering 
education adequacy has been used in prior studies across the states (and an assessment of its 
strengths and weaknesses); whether successful schools could be identified for California, and if 
yes, what we might learn from them; and the implications of these findings for the overall 
concept of adequacy. The successful schools approach is one of four predominant methods for 
considering and attempting to measure education adequacy. A simplified summary of its possible 
primary advantage over other approaches seems to be the argument that the most effective way 
to determine the cost of education needed to reach a specified level of educational outcomes is to 
identify schools (districts) achieving these desired outcomes and to determine how much they are 
spending. 

This line of reasoning is compellingly straightforward, but at the same time overly simplistic. A 
number of concerns associated with this approach to determining adequacy are described in 
Chapter 2. One serious concern is the dilemma of exactly how successful schools will be 
selected. On the one hand, we can select schools already achieving the state’s specified outcome 
goals (e.g., a score of 800 measured by the Academic Performance Index, or API). This seems to 
meet the criterion of selecting schools that have actually achieved specified goals as the basis for 
costing out what is required to do this. However, these schools have the substantial disadvantage 
of generally being very unrepresentative of the rest of the schools in the state in regard to their 
percentages of special populations of students such as those in poverty or who are English 
learners (ELs). 

Another option is to follow the selection methods employed in this study by defining successful 
schools as those “beating the odds” in the sense that they are consistently performing at a much 
higher level than expected given the composition of students they serve. This has the advantage 
of resulting in a pool of successful schools (termed “beating the odds” given the approach to 
their selection) that is more representative of the state in regard to the mix of students served. 
However, the disadvantage here is that only a small number of schools seem to be outperforming 
their counterparts over the time period under analysis. In addition, it is not guaranteed that these 
schools will also be meeting an absolute state- specified outcome goal. Eor example, an API of 
800 can be considered a desirable goal for California schools. Yet of the 103 BTO schools 
selected for this study, only 41 were at or above the state school accountability target of 800. 

Thus, although these schools are arguably excellent models of resource allocation in that they are 
performing at a much higher level than their similar counterparts, they do not necessarily reveal 
what is needed to meet the state’s specified outcome standard, which is the primary objective of 
an adequacy determination. Eurthermore, only a small number of schools are consistently 
performing at a higher level than what is expected for them. 

This does not mean that there is nothing to be learned from the identification and analysis of 
successful schools. As discussed in Chapter 1, however, the definition of these schools should 
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not be simply those already meeting the state standard. Due to the very atypical demographics of 
these schools, as shown in Exhibit 1.1, fairly little can be learned from these schools that would 
be generalizable to the average school in the state. 

Therefore, we used a BTO approach to successful school selection, as described in Chapter 3. 
This chapter addresses the third research question for this study: how successful schools might 
be identified in the state. However, for the reasons cited above we have not identified these 
schools for the purpose of attempting to define adequacy. Just because these schools are 
successful does not mean that the resource patterns observed at these schools can be extrapolated 
to other schools with the assumption that this is the mix of resources that will also lead them to 
success. 

Rather, we see several advantages to identifying and further examining these schools. Although 
these BTO schools are not necessarily all meeting the specified state outcome standard, and 
although there are only about 100 schools in the state, they all consistently perform at very high 
levels in relation to like schools, and also perform well in an absolute sense in relation to all 
schools. As shown in Exhibit 6.1, a number of the BTO schools are at or above the state school 
achievement target of 800 API and at the same time have a high poverty level. 

Exhibit 6.1. Distribution of APi and Percentage of Students in Poverty for Aii 103 BTO Schools 




While identifying and analyzing schools with a broad mix of student characteristics that have 
been consistently outperforming similar schools may not be well suited to directly answering the 
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adequacy question, it does provide a basis for further considering the extent to which it is indeed 
variations in the quantities and characteristics of the resources employed (as we currently 
measure them) that really make a difference in regard to school success. Perhaps it is not 
resources at all, or if it is resources perhaps it is due to characteristics related to those resources 
that do not lend themselves to measurement. 

Chapter 4 addresses these questions. What resource differences are observed at BTO schools? Is 
there evidence that BTO schools use their resources more efficiently? Can we predict academic 
performance by levels of resources and types of students enrolled? In summary, we found fairly 
few and relatively small resource differences between BTO, low performing (LP), and other 
schools in the state. Furthermore, the level of resources and student characteristics do not explain 
the high performance level observed in BTO schools. BTO schools do appear to have a more 
efficient mix of personnel resources in terms of estimated optimal levels of teacher education and 
experience, but as mentioned, this analysis is really only instructive in the sense that these 
schools have discretion to choose the education and experience levels of their staff. 

In fact, no school (or district, for that matter) has complete control over this mix of teacher 
attributes, but some schools may have much more control than others in regard to how much 
latitude they have in hiring and firing teachers. This may be an important variable in better 
understanding and being able to statistically explain varying levels of school success. Another 
may be the number of years a principal has been at a given school, or measures of teacher 
turnover at a school, and perhaps teacher attendance. These are possible measures of school 
stability and climate that may be as important or as, or even more important than, the resource 
quantities and attributes we currently measure. 

Given that they are producing much higher educational outcomes than their counterpart schools 
with apparently comparable levels of resources, the BTO schools identified in this study are 
arguably among the most efficient in the state. As the current state and federal accountability 
movement appears to be a clear call to arms to improve student achievement, it would seem 
imperative to identify schools such as those included in this study to further investigate the 
factors related to their success. 

This was the focus of the phone interviews for this study and the resulting analyses, which is 
summarized in Chapter 5. If the primary differences in the BTO schools are not resource related, 
or at least not related to resource measures currently being collected by the state, what 
differences in these schools do appear associated with their unusual success? Chapter 5 showed 
that a number of factors appear to be important contributors to school success. In addition, it 
argued that school success is much less of a clear recipe that results from a clearly specified mix 
of ingredients than a confluence of factors that under the right conditions can produce striking 
results. 

At the same time, it may be a mistake to conclude that school success is so idiosyncratic that 
there are no common elements or things that can be measured and perhaps replicated. Chapter 5 
summarizes some of these common themes and ties them to what we know about school success 
from the literature. The major factors identified by the BTO respondents for this study were: 1) 
existence of high-quality teachers and staff; 2) implementation of a standards-based curriculum; 
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and/or 3) coherent instruction. Although these overarching themes are instructive, they are 
perhaps less directive than we would like in terms of providing guidance in regard to what policy 
makers might do to replicate the results of these schools statewide. The complexity of what 
makes a school unusually successful, and the possible difficulties of replicating these results, is 
perhaps best summarized by one of the respondents in this study: 

You’re looking for the recipe for how to get a school to be successful but I think running a 
school is an art form. You can give a recipe for making a film but it would not make it the 
best film in town. You can follow the whole recipe but you miss that other ingredient, which 
is the artistic part of it. - BTO school principal 

At the same time, although the linkage between the existence of high-quality teachers and staff 
cited above and school success seems somewhat obvious, the findings from this report suggest 
that such staff can be attracted to schools with high percentages of students with special needs 
(e.g., students in poverty and with limited English). A major variable associated with attracting 
teachers to these schools may be the creation of an environment in which they believe they have 
a chance to be successful. Some resource considerations related to this may be stable leadership, 
district support, and discretion at the local level in regard to being able to attract and retain other 
high-quality teachers and staff and to remove those who prove to be ineffective. 

From an overall adequacy perspective, these findings seem to challenge the basic underlying 
premise that the primary element that is lacking in regard to realizing state outcome goals is 
directly related to the quantities (or even the attributes) of educational resources. Simply adding 
more resources may not be likely to make a difference in regard to school performance. This 
would suggest a somewhat different conceptualization of the adequacy question than has been 
commonly employed. 

While it still may be that resources considered broadly are what make the primary difference in 
schools, it seems we need to broaden our conceptualization and measures of these resources 
beyond quantities. Undoubtedly there are certain minimum levels of resources that are 
imperative to school success. Beyond this, however, we may need to broaden this perspective to 
begin specifying adequate conditions for schools’ success. To examine this further, we may need 
resource measures that, at least in California, we do not currently have. For example, we do not 
have measures of the stability of leadership and instructional staff at the school (e.g., number of 
years at the school). We do not know the degree to which is there latitude for schools to select, 
retain, and remove teachers as needed to ensure a “quality” staff. We have insufficient measures 
ensuring district support for high needs schools — e.g., ensuring that they have at least equal 
resources in comparison to all schools in the district, and a higher degree of selectivity in regard 
to the attributes of these teachers than schools with lower percentages of at-risk students. We 
also lack measures such as teacher and administrator absenteeism that might provide an 
indication of the overall atmosphere and day-to-day stability at the school. 

In summary, although the successful schools approach to defining education adequacy seems 
deficient in many fundamental ways, identifying and analyzing BTO schools may provide 
insights into our overall conceptualization of educational adequacy. Its basic underlying premise 
as it has been largely defined and applied is that we simply need a better understanding of the 
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levels of resources needed to reach a specified educational outcome standard. The analyses in 
this report suggest that, at least for the pool of schools realizing this level of success at a much 
greater rate than their counterparts, traditional resource measures do not seem to be what are 
making the difference. 

This does not lead to the conclusion that resources do not matter. All of these schools do have 
resources at a certain specified level; none would likely say that they could continue to perform 
as well with less, and most would probably argue for more. In addition, one of the primary 
factors that they attribute to their success is clearly resource based: “a high quality staff.” The 
differences between the staff at these schools and other schools, or even those schools included 
in this study because they are performing much worse than expected, are not captured well by 
the types of measures currently collected by the state. Perhaps existing adequacy frameworks 
would benefit from considering more broadly the mix of school-wide staff attributes, as well as 
counts of staff and non-personnel resources, needed in a school to be truly adequate for success. 
The state can further this agenda by more comprehensive data collection in regard to the broader 
sets of attributes and performance measures that are needed to better understand the full resource 
implications of schools’ success. 
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